1235 Commits (df304eb016f440ca01a597ade98180a87a9a407a)

Author	SHA1	Message	Date
Broque Thomas	df304eb016	AcoustID scanner: handle multi-value artist credits ... Discord report (Foxxify): the AcoustID scanner repair job flagged multi-artist tracks as Wrong Song because AcoustID returns the FULL credit ("Okayracer, aldrch & poptropicaslutz!") while the library DB carries only the primary artist ("Okayracer"). Raw SequenceMatcher similarity scored ~43% — well below the 60% threshold — so the scanner created a finding even though the audio was correct. User couldn't fix without lowering the global artist threshold to ~30% (which would let real mismatches through). # Fix Extended the shared `core/matching/artist_aliases.py::artist_names_match` helper (originally lifted for #441) with credit-token splitting. When the actual artist string contains common separators — - punctuation: `,` `&` `;` `/` `+` - keywords (whitespace-bounded): `feat.` `ft.` `featuring` `with` `vs.` `x` — the helper splits into individual contributors and checks each against the expected artist. Primary-in-credit cases now resolve at 100% instead of 43%. Two pattern groups because punctuation separators don't need surrounding whitespace, but keyword separators MUST be whitespace-bounded — otherwise we'd split artists with `x` / `with` etc. in their names ("JAY-X" → "JAY-" / "" issue). Composes with the existing alias path: cross-script multi-artist credits ("Hiroyuki Sawano" expected, "澤野弘之, FeaturedJp" actual) work via alias-token-against-credit-token compare. # Wire-in Scanner at `core/repair_jobs/acoustid_scanner.py:202` replaces the raw `SequenceMatcher` call with `artist_names_match`. Pass RAW artist strings (not pre-normalised by `_normalize`) so the splitter can recognise separators — `_normalize` strips ALL punctuation, which destroyed the very tokens the splitter needs. The AcoustID post-download verifier (`core/acoustid_verification.py`) already routes through `_alias_aware_artist_sim` which calls the same helper — gets the multi-value benefit automatically without a separate wire-in. # New `split_artist_credit` exported helper Pure-function helper for callers who want token-level access to the credit list (debugging, UI, future per-token enrichment). Same splitter logic, exposed as a top-level function. # Tests added (14) `tests/matching/test_artist_aliases.py` (+11): - `TestSplitArtistCredit` — parametrised across 12 credit-string formats (comma, ampersand, semicolon, slash, plus, feat./ft./ featuring, with, vs., x, single-token, empty), drops empty tokens, strips per-token whitespace - `TestMultiValueCreditMatching` — reporter's exact case (Okayracer in 3-artist credit → 100%), primary in middle/end of credit, genuine-mismatch still fails, single-token actual falls through to direct compare, multi-value composes with aliases, threshold still respected `tests/test_acoustid_scanner.py` (+3): - Reporter's case end-to-end through `_scan_file` — fingerprint 99% / title 100% / multi-artist credit → no finding created - Genuine artist mismatch still creates finding (no false suppression of real mismatches) - `JobResultStub` minimal scaffold for the integration tests # Verification - 14 new tests pass (49 helper + 5 scanner total in their files) - 110 matching + scanner tests pass total - 2584 full suite passes (+25 from baseline 2559) - Ruff clean - Reporter's exact case (Okayracer in `Okayracer, aldrch & poptropicaslutz!`) now scores 100% match → no Wrong Song flag	1 month ago
Broque Thomas	80cf16339c	Deezer cover art: upgrade CDN URL to 1900×1900 (was embedding 1000×1000) ... Discord report (Tim): downloaded cover art via Deezer metadata source came out visibly blurry in Navidrome / on phones — large displays exposed the limited resolution. # Cause Deezer's API returns `cover_xl` URLs at 1000×1000. The underlying CDN actually serves up to 1900×1900 by rewriting the size segment in the URL path (same trick the iTunes mzstatic + Spotify scdn upgrades already use). SoulSync wasn't doing the rewrite — every Deezer-sourced cover got embedded at 1000×1000 regardless of how much higher resolution the CDN had available. # Verified empirically ``` $ for size in 1000 1400 1800 1900 2000; do curl -I "...{size}x{size}-..."; done 1000: 200 OK 106 KB 1400: 200 OK 198 KB 1800: 200 OK 331 KB 1900: 200 OK 371 KB 2000: 403 Forbidden ``` 1900 is the safe ceiling. Above that the CDN returns 403. CDN serves source-native bytes when source < target (smaller-source albums get same bytes whether we ask for 1000 or 1900), so asking for 1900 universally is safe. # Fix New `_upgrade_deezer_cover_url(url, target_size=1900)` helper in `core/deezer_client.py`. Pure function, mirrors the `_upgrade_spotify_image_url` pattern that already lives in `core/spotify_client.py`. Defensive on every input shape: - Empty / None → returned as-is - Non-Deezer URL (no `dzcdn`) → returned as-is - No size segment in URL → returned as-is - Already at/above target → returned as-is (idempotent, never downgrades) Applied at both cover-download sites: - `core/metadata/artwork.py::download_cover_art` — auto post-process flow. Mirrors the existing iTunes mzstatic upgrade right above it. - `core/tag_writer.py::download_cover_art` — enhanced library view's "Write Tags to File" feature. # Scope discipline - Helper applied at the DOWNLOAD boundary, not the source extraction point in `deezer_client.py`. Means cached entries in the metadata cache + DB row `image_url` columns keep the original 1000×1000 URL Deezer's API returned. Future CDN behavior changes only affect the download path, not stored data. - Pre-existing `prefer_caa_art` toggle (Settings → Library → Post-Processing) untouched — orthogonal workaround for users who want even higher quality (MusicBrainz Cover Art Archive, often 3000×3000+). - iTunes / Spotify upgrade paths untouched — they already worked. # Tests added (16) `tests/metadata/test_deezer_cover_url_upgrade.py`: - Standard upgrade: default target 1900 on cover URL, alternate dzcdn host (`e-cdns-images.dzcdn.net` vs `cdn-images.dzcdn.net`), artist picture URLs (same path pattern), 500×500 source upgrades too - Custom target size: smaller target = no-op (never downgrade), larger target works - Idempotent: already at/above target returned unchanged - Defensive on non-Deezer URLs: parametrised across 5 hosts (Spotify scdn, iTunes mzstatic, MB CAA, Last.fm, random) — all returned untouched - Defensive on malformed Deezer URL (no size segment) → returned as-is - Empty / None handling # Verification - 16/16 helper tests pass - 560/560 metadata + imports tests pass (no regression) - 2559 full suite passes - Ruff clean	1 month ago
Broque Thomas	80e9398e16	WHATS_NEW: cross-script artist names no longer quarantine files (#442)	1 month ago
Broque Thomas	c02d51d60d	Plex: trigger_library_scan + is_library_scanning use auto-detected section — fixes #535 ... # Bug Plex servers with the music library named anything other than "Music" (Música, Musique, Musik, Musica, 音乐, موسيقى, etc.) hit this error after every import cycle: soulsync.plex_client - ERROR - Failed to trigger library scan for 'Music': Invalid library section: Music soulsync.web_scan_manager - ERROR - Failed to initiate PLEX library scan via web Side effect: `wishlist.processing` kept reporting "Missing from media server after sync" for tracks that DID import correctly, so they got perpetually re-added to the wishlist. # Root cause `_find_music_library` correctly auto-detects the music section by `section.type == 'artist'` and stores it on `self.music_library` — works for any locale because the type is language-neutral. Read methods (`get_artists`, etc.) route through `_get_music_sections` which returns `[self.music_library]`, so they never had the bug. But `trigger_library_scan` and `is_library_scanning` ignored `self.music_library` and called `self.server.library.section(library_name)` directly with the hardcoded `"Music"` default. `server.library.section('Music')` raises `NotFound` on any server whose section isn't literally named "Music". # Fix Both methods now prefer `self.music_library` first, fall back to literal `library_name` lookup only when auto-detection hasn't populated the cached reference (test fixtures, edge cases). `is_library_scanning`'s activity-feed match also corrected to filter by the resolved section's actual title — the prior code matched `library_name.lower() in activity_title.lower()` which defaults to "music" and would never match activities for non-English sections. `trigger_library_scan`'s success log line now surfaces the actual section title (`Música`) instead of the unused `library_name` default ("Music") — confusing when debugging on non-English servers. # Tests added (13) `tests/media_server/test_plex_non_english_section_name.py`: - `test_uses_auto_detected_section_regardless_of_locale` — parametrised across 6 locale variants (Música, Musique, Musik, Musica, 音乐, موسيقى). Each verifies trigger_library_scan calls the auto-detected section's `update()`, NOT a literal-name fallback. Stub raises AssertionError on `server.library.section()` so a regression that re-introduces the fallback fails loudly. - `test_falls_back_to_literal_lookup_when_no_auto_detection` — backward compat: music_library=None → literal lookup as before. - `test_explicit_library_name_arg_used_only_when_no_auto_detection` — auto-detected wins over explicit kwarg when both available. - `test_logs_correct_section_label_on_success` — log line surfaces resolved section title. - 4 symmetric tests for is_library_scanning covering refreshing-attr check, activity-feed title match, no-match for unrelated sections, fallback path. # Verification - 13 new tests pass - 84/84 media_server tests pass (no regression in the existing Plex / Jellyfin / Navidrome suite) - 2458 full suite passes (+13 from baseline) - Ruff clean	1 month ago
Broque Thomas	402d851cac	Deezer search: drop advanced-syntax at endpoint, free-text + rerank wins ... Live-API verification revealed advanced-syntax queries hurt more than they help on this endpoint. Switching the import-modal Deezer search back to free-text + local rerank. # What live testing showed Hit Deezer's public API with both query forms for the issue #534 case (`Dirty White Boy` + `Foreigner`): **Free-text (`q=Dirty White Boy Foreigner`):** - Returns 21 results - Real Foreigner Head Games studio cut at #1 - Live versions at #2-10 - Karaoke / cover variants at #11-15 **Advanced (`q=track:"Dirty White Boy" artist:"Foreigner"`):** - Returns 12 results - "(2008 Remaster)" at #1 — canonical Head Games cut MISSING from top 8 entirely - Live + alt-album versions follow Advanced syntax DOES filter karaoke at the API level (none in the 12-result set vs. 5 at positions 11-15 in free-text), but it has its own ranking bias that surfaces remasters / "Best Of" cuts ahead of the canonical recording. Net regression for the user- facing goal. # Fix 1. Endpoint reverts to free-text query with local rerank applied. 2. Local rerank gains "remaster" / "remastered" / "reissue" patterns under VARIANT_TAG_PATTERNS (soft 0.4× penalty — user may want them but they shouldn't outrank the original). 3. Client kwarg support (`track=` / `artist=` / `album=`) preserved for future opt-in callers (e.g. exact-match flows where API- level filtering matters more than ranking). # Verified end-to-end against live Deezer API Re-ran the exact #534 case through the live API + new rerank. Top 15 results post-rerank: 1. Dirty White Boy — Foreigner — Head Games ← REAL CUT AT TOP 2-10. Various Live versions 11-15. Karaoke / cover / tribute variants ← BURIED Real Foreigner Head Games studio cut at #1, exactly the user's ask. # Tests - `test_relevance.py` — variant tag patterns extended; existing tests still pass (50 tests). - `test_search_match_endpoints.py::test_joins_track_and_artist_into_free_text_query` — replaces `test_passes_track_and_artist_as_kwargs`; verifies endpoint sends free-text join, NOT field-scoped kwargs (the prior test asserted the wrong direction now). - Karaoke-burying assertion at the endpoint still pins the user-visible behaviour. - Client kwarg path tests untouched (still pin advanced-syntax construction for future opt-in callers). # Verification - 75 relevance + endpoint + query tests pass - 2445 full suite passes - Ruff clean - Live Deezer API shows real cut at #1 post-rerank	1 month ago
Broque Thomas	59992d42a8	Deezer search: free-text fallback when advanced query returns 0 ... Defensive followup to the relevance fix. Deezer's advanced search syntax (`artist:"X"`) is documented as substring match, but in practice it's brittle on artist name variants ("Foreigner [US]", "The Foreigner") and on tracks indexed under non-canonical title spellings. When the advanced query returns nothing, we'd previously land at "No matches" — a regression vs. pre-fix behaviour where free-text would have returned a less-relevant but non-empty set. Fix: when the advanced query returns 0 results AND the caller used field-scoped kwargs, fall back to a free-text join of the same kwargs and re-query. Caller-side rerank still tightens whatever the fallback returns, so the worst-case post-fix behaviour is the pre-fix behaviour — never strictly worse. Pulled the cache + parse + store dance into a private helper (`_search_tracks_with_query`) so the orchestration can call it twice (advanced → fallback) without code duplication. Single API call when the advanced query has results — no wasted requests. Diagnostic logger.debug fires when the fallback triggers so we can see in production whether it's happening (and to which queries). # Tests added (4) - `test_falls_back_to_free_text_when_advanced_empty` — advanced query returns 0, free-text returns hits; client returns the free-text hits + both API calls fire. - `test_no_fallback_when_advanced_query_has_results` — single hit on advanced query → no second API call. - `test_no_fallback_when_legacy_free_text_call` — legacy callers already exhausted the only path; empty result is final. - `test_no_fallback_when_query_unchanged` — empty kwargs path doesn't trigger the fallback branch (used_advanced=False). # Existing tests updated The 4 prior `TestSearchTracksQueryWiring` + `TestSearchTracksCacheKey` tests were stubbing `_api_get` to return empty `{'data': []}` and asserting `assert_called_once`. With the new fallback, those stubs trigger a second API call and the assertions break — even though the FIRST call construction is what the tests cared about. Updated the stubs to return one fake hit so the fallback doesn't fire, and switched to `call_args_list[0]` for first-call inspection. # Verification - 18/18 deezer query tests pass (14 prior + 4 new) - 2445 full suite passes (+4 from prior commit) - Ruff clean	1 month ago
Broque Thomas	1cc37081a6	Fix Deezer search relevance — issue #534 ... # Background User reported (#534) that the import-modal "Search for Match" dialog returned irrelevant results when Deezer was the metadata source. Searching `Dirty White Boy` + `Foreigner` returned 5+ karaoke / "originally performed by" / "in the style of" / "re-recorded" / tribute-band results ranked above the actual Foreigner studio cut from Head Games. User had to scroll past the junk every time, or fall back to iTunes search which is much slower. # Root cause — two layers 1. **Endpoint joined `track + artist` into free-text query.** `/api/deezer/search_tracks` was passing `q=Dirty White Boy Foreigner` to Deezer's `/search/track` API. Deezer fuzzy-matches that string across title / lyrics / artist / album / contributors and orders by global popularity — anything that appears across many compilations outranks the canonical recording. 2. **No local rerank.** None of the search-modal endpoints applied any post-filtering. Deezer's API order shipped straight to the user. # Fix — same architectural shape Cin would build ## Layer 1: field-scoped query at the client boundary `core/deezer_client.py::search_tracks()` now accepts optional `track`, `artist`, `album` kwargs. When provided, builds Deezer's advanced search syntax: `q=track:"X" artist:"Y" album:"Z"`. Massive relevance improvement because each term matches the right field instead of fuzzy-matching everywhere. Backward compat preserved: legacy free-text `query=` callers still work unchanged. Field-scoped path takes precedence when both are provided. Empty input fast-fails without an API call. Embedded double-quotes stripped (Deezer's syntax has no escape mechanism). ## Layer 2: provider-neutral relevance reranker New `core/metadata/relevance.py` module — pure-function rerank over the canonical `Track` dataclass. Composable scoring: - **Cover/karaoke patterns** (multiplier 0.05, effectively buries): matches "karaoke", "originally performed by", "in the style of", "made famous by", "tribute", "vocal version", "backing track", "cover version", "re-recorded", "cover by", etc. across title, album, AND artist fields. Catches the screenshot's exact junk: artist credits like "Pop Music Workshop" / "The Karaoke Channel" / "Foreigner Tribute Band". - **Variant tags** (multiplier 0.4): live / acoustic / demo / instrumental / remix / radio edit / club mix etc. — softer penalty since the user MAY want them. Skipped entirely when the expected_title contains the same tag (so searching "Track (Live)" still ranks Live versions first). - **Exact artist boost** (multiplier 1.5): primary artist exactly matches expected_artist after normalisation. Single strongest signal for "this is the canonical recording". - **Title + artist similarity** via SequenceMatcher (parentheticals + punctuation stripped before comparison). - **Album-type weighting**: album=1.0 > single/ep=0.85 > compilation=0.7. Compilations are more likely tribute / karaoke repackages. Each component is a standalone function so tests pin them individually without standing up the full pipeline. ## Wired at three search-modal endpoints - `/api/deezer/search_tracks` — uses both layers (field-scoped query + rerank). - `/api/itunes/search_tracks` — uses rerank only (iTunes API has no advanced-syntax search, but karaoke / cover variants still leak through and need the local penalty). - `/api/spotify/search_tracks` — already builds field-scoped `track:X artist:Y` query; rerank added as the consistency safety net so all three sources behave the same from the user's perspective. Other Deezer call sites (matching engine, watchlist scanner, auto-import single-track ID) deliberately not touched in this PR — they have their own elaborate scoring pipelines tuned to their specific contexts and aren't surfacing the user-reported issue. Per Cin: "don't refactor beyond what the task requires." # Tests 71 new tests across 3 files: - `tests/metadata/test_relevance.py` (50 tests) — every scoring component pinned individually + the issue #534 screenshot reproduced as a regression test (real Foreigner cut wins after rerank, karaoke variants drop to bottom). - `tests/metadata/test_deezer_search_query.py` (14 tests) — advanced-syntax query construction, field-scoped wiring at the client boundary, free-text path unchanged, kwargs win when ambiguous, limit clamping, cache key consistency. - `tests/imports/test_search_match_endpoints.py` (7 tests) — end-to-end through Flask test client: Deezer endpoint passes kwargs not joined query; karaoke buried at bottom for all three sources; legacy query param still works without rerank. # Verification - 2441 full suite passes (+71 from baseline 2370) - 0 failures (the prior watchdog flake fix held) - Ruff clean across all changed files - JS parses clean (`node -c webui/static/helper.js`) # Architectural standards followed - **Logic at the right boundary.** Query construction lives in the client (every caller benefits from one change). Rerank lives in a neutral module (`core/metadata/relevance.py`) over the canonical `Track` dataclass — works for any source, not Deezer- specific. - **Explicit > implicit.** Every scoring rule has its own named function. Pattern tables are module-level constants tests can introspect. - **Scope discipline.** Audited every Deezer search call site; fixed the user-reported one + the consistent siblings. Did NOT speculatively normalise every Deezer call across the codebase. - **Backward compat.** Free-text `query=` callers untouched. Kwargs added to existing client method signature with safe defaults. - **Tests pin contract at correct boundary.** Pure-function rerank tests don't mock anything; client-query tests stub at `_api_get`; endpoint tests run through the real Flask app.	1 month ago
Broque Thomas	abab663eb7	Auto-import: album duration = album total + conservative re-import UPDATE path ... Two pre-existing parity gaps in `record_soulsync_library_entry` that the prior parity commits left untouched. Both close real holes between auto-import writes and what the soulsync_client deep scan would have produced. # Gap 1: Album duration was the first-imported track's duration `record_soulsync_library_entry` is called once per track. The album INSERT only fires for the FIRST track of a new album (subsequent tracks find the album row already exists). The INSERT was passing `duration_ms` — `track_info["duration_ms"]` — as the album's `duration` column. That's the duration of one track, not the album total. Compare to `SoulSyncAlbum.duration` in soulsync_client which is `sum(t.duration for t in self._tracks)`. Fix: - Worker computes `album_total_duration_ms = sum(...)` across every matched track and threads it onto context as `album.duration_ms`. - side_effects reads that value (or falls back to the per-track duration for legacy non-auto-import callers) and writes it as the album row's `duration`. # Gap 2: Re-imports of the same artist/album were insert-only When the SELECT-by-id or SELECT-by-name found an existing soulsync artist or album row, the function skipped completely — no UPDATE path. Meant: artist genres / thumb / source-id reflected ONLY whatever the FIRST imported album supplied, never refreshing as more albums by that artist landed. Ten more imports later, the artist row still held whatever the first random import wrote. Conservative fix: when an existing row matches, run an UPDATE that fills only the columns whose current value is NULL or empty. Never overwrites populated values — protects manual edits + enrichment-worker writes the same way the scanner UPDATE path preserves enrichment columns. Implementation note: the empty-check happens in Python, NOT SQL. Initial pass tried `COALESCE(NULLIF(col, ''), NULLIF(col, 0), ?)` but SQLite's `NULLIF(text_col, 0)` returns the original text value instead of NULL — different types, no coercion. So the SQL-only conditional was unreliable on text columns. New helper does `SELECT cols FROM table WHERE id`, compares each column in Python, and emits UPDATE clauses only for the ones that need filling. Allowlist defense: f-string column names go through `_SOULSYNC_FILLABLE_COLUMNS` validation before interpolation. Misuse adding new columns without an allowlist update fails closed (logger.debug + skip). # Tests added (4) - `test_album_duration_uses_album_total_not_single_track` — album with single-track context carrying explicit `album.duration_ms = 2_500_000` writes 2_500_000 to the album row, not the per-track 200_000 fallback. - `test_re_import_fills_empty_artist_fields` — first import lands artist with empty thumb + empty genres; second import for same artist with thumb + genres present updates the existing row. - `test_re_import_does_not_clobber_populated_artist_fields` — first import writes rich genres + thumb; second import with worse / different metadata leaves the existing row untouched. - `test_re_import_fills_empty_source_id_when_missing` — first import had no source artist ID; second import does — fills the empty `spotify_artist_id` column on the existing row. # Verification - 10/10 side-effects tests pass (including 4 new + 4 from prior parity commit + 2 history/provenance) - 217 imports tests pass (no regression) - 2369 full suite passes (+4 from prior, +22 PR-total from baseline 2347) - 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`, passes in isolation, unrelated) - Ruff clean	1 month ago
Broque Thomas	f628009ab4	Auto-import: aggregate GENRE tags onto artists row + harden ISRC/MBID types ... Cin pre-review followup. Two small parity gaps the prior commits left open: # 1. Genre tags land on the standalone artists row `soulsync_client._scan_transfer` aggregates the GENRE tag across every track in an album and surfaces it on `SoulSyncAlbum.genres` (which the DatabaseUpdateWorker writes to the artists+albums row). Auto-import was hardcoding `'spotify_artist': {'genres': []}` so the imported artists row landed with empty genres — felt hollow compared to a Plex/Jellyfin scan, which both pull genres from their respective APIs. Fix: - `_read_file_tags` now reads the GENRE tag (mutagen easy mode handles MP3/FLAC/M4A consistently; some files carry multiple genres so it's always returned as a list). - `_process_matches` aggregates genres from each matched file's tags into a deduped insertion-order list. Dedup is case-insensitive but preserves original casing — so "Hip-Hop, Rap, Trap" reads naturally in the JSON column instead of "hip-hop, rap, trap". - Worker context's `spotify_artist['genres']` carries the aggregated list, which `record_soulsync_library_entry` already filters via `core.genre_filter.filter_genres` and writes to the artists row. # 2. Defensive str() cast for ISRC + MBID `_build_album_track_entry` already coerces ISRC + MBID to string today (via `str(isrc) if isrc else ''`). But if a future metadata-source client returns int / None for either ID, the worker would propagate the wrong type and side_effects.py's `.strip()` would AttributeError. Cheap insurance: explicit `str()` cast in the worker before assignment to track_info. Future-proofs against client drift. # Tests added (3, in test_auto_import_context_shape.py): - `test_context_aggregates_genres_from_track_tags` — multi-file album with overlapping genre lists produces deduped, insertion- ordered, original-case-preserved result. Stubs `_read_file_tags` with monkeypatch so we don't need real audio. - `test_context_genres_empty_when_no_tags` — files without GENRE tag → empty list. Standalone library write handles gracefully (genres column stays empty / NULL). - `test_context_isrc_mbid_coerced_to_string` — hostile types (int 12345678, None, int 999) coerced to safe strings before reaching track_info. # Verification - 14/14 context-shape tests pass (11 prior + 3 new) - 213 imports tests pass (no regression) - 2365 full suite passes (+3 from prior, +18 PR-total) - 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`, passes in isolation) - Ruff clean	1 month ago
Broque Thomas	ec7da89434	Auto-import: surface artist source-id from metadata search response ... Cin pre-review followup to the standalone library parity commit. The prior commit fixed `spotify_artist['id']` from the wrong copy-paste value (`identification['album_id']`) to read from `identification['artist_id']`, but the identification dict produced by `_search_metadata_source` and `_search_single_track` never set `artist_id` — both extracted artist NAME from the search response and discarded the source ID sitting right next to it. Net effect of the prior commit: artists row source-id stayed NULL, just for a more honest reason than before. Now properly extracted: - `_search_metadata_source` reads `best_result.artists[0]['id']` alongside the artist name and returns it on the identification dict as `artist_id`. - `_search_single_track` does the same for single-track identification. - `_identify_single`'s tag-based-confidence path forwards `result.get('artist_id')` so the artist source-id propagates even when high-confidence local tags override the search result's name. Result: identification dict now carries `artist_id` whenever the metadata source returned an artist with an ID. The worker context already plumbs it onto `spotify_artist['id']` and `spotify_album['artists'][0]['id']`, so the standalone library write finally populates `<source>_artist_id` on the artists row. Tests added (3, in `test_auto_import_context_shape.py`): - `test_context_artist_id_uses_identification_artist_id` — when the identification dict carries `artist_id`, context propagates it onto `spotify_artist['id']` AND `spotify_album['artists'][0]['id']`. Pins that the prior copy- paste bug (artist['id'] = album_id) doesn't return. - `test_context_artist_id_is_empty_when_identification_missing_it` — fallback case (filename-only identification): context gets empty string, NOT album_id. Honest failure mode. - `test_search_metadata_source_extracts_artist_id_from_dict_artist` — black-box test of `_search_metadata_source`: feed it a spotify-shaped result with `artists[0]['id']` and verify identification dict carries it forward. Verification: - 11/11 context-shape tests pass (8 prior + 3 new) - 210 imports tests pass (no regression) - 2362 full suite passes (+3 from prior commit, +15 PR-total) - 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`, passes in isolation) - Ruff clean	1 month ago
Broque Thomas	8493be207e	Auto-import: SoulSync standalone library writes server-quality rows ... # Background SoulSync standalone is meant to be a full replacement for Plex / Jellyfin / Navidrome — files imported via auto-import (or any other import path) should land in the database with the same field richness a media-server scan would write. They weren't. # Gaps fixed The auto-import worker built a context dict for each track and handed it to `_post_process_matched_download` (the same callback the regular download flow uses). That dict was missing three things downstream needed: 1. **No `source` field anywhere.** `record_soulsync_library_entry` reads `get_import_source(context)` to pick the source-aware ID columns (`spotify_track_id` / `deezer_id` / `itunes_track_id` / etc.) on the artists / albums / tracks rows. With no source, the resolver returned an empty string → `get_library_source_id_columns("")` returned an empty dict → the `UPDATE tracks SET <source>_id = ?` blocks were silently skipped. Result: every auto-imported track landed with NULL on every source-id column. Watchlist scans (which match by stable source IDs to detect "this track is already in library") couldn't recognise these rows and would re-download them on the next pass. 2. **No `_download_username='auto_import'`.** Both `record_library_history_download` and `record_download_provenance` default to "Soulseek" when no `username` is in the context. Every staging-folder import was being labelled as a Soulseek download in library history + provenance — false signal in the UI. 3. **No per-recording IDs (`isrc`, `musicbrainz_recording_id`) on track_info.** The Navidrome scanner already writes `musicbrainz_recording_id` directly to the tracks row when present. Picard-tagged libraries always carry MBID; metadata sources (Spotify via MusicBrainz enrichment, Deezer, etc.) carry ISRC. Auto-import had access to both via the metadata-source response but didn't propagate them — so the soulsync row went in with NULL on both columns. # Changes **`core/auto_import_worker.py` — `_process_matches`:** - Top-level `'source': source` (from `identification['source']`) - `'_download_username': 'auto_import'` - `track_info['isrc']`, `track_info['musicbrainz_recording_id']` — pulled from the per-track payload returned by the metadata source - `track_info['album_id']` — back-reference so source-aware ID resolution works on sources whose API nests album under `track.album.id` rather than `track.album_id` - `spotify_artist['id']` now correctly carries the artist's source ID (was `identification['album_id']`, a copy-paste bug from the original implementation that made artist-id resolution fall back to fuzzy matching) - `spotify_album['artists'][0]['id']` carries artist source ID for the same resolution path **`core/imports/side_effects.py`:** - `record_library_history_download` source_map: add `"auto_import": "Auto-Import"` — tags imported tracks correctly - `record_download_provenance` source_service: add `"auto_import": "auto_import"` — provenance shows real source - `record_soulsync_library_entry` track INSERT: now includes `musicbrainz_recording_id` + `isrc` columns (matches `insert_or_update_media_track`'s shape for Navidrome / Plex / Jellyfin scans). Both default to NULL when not present. # Behavior preserved - Files still land in the same library template path (no path-build change) - Other media-server flows (Plex / Jellyfin / Navidrome users) unaffected — `record_soulsync_library_entry` still gates on `get_active_media_server() == "soulsync"`. Auto-import on those servers continues to drop the file in the library folder + emits `batch_complete` for the scan-trigger automation, same as before. - Direct downloads (search → Download button) unaffected — they already passed `source` + `username` correctly. # Tests added `tests/imports/test_auto_import_context_shape.py` (8 tests, new file): - Worker context carries `source` for every metadata source (parametrised across spotify / deezer / itunes / discogs) - `_download_username='auto_import'` set unconditionally - ISRC + MBID propagate from track payload to track_info when present - ISRC + MBID default to empty string when absent (downstream normalises to NULL at write time) - track_info includes album-id back-reference `tests/imports/test_import_side_effects.py` (4 new tests + 2 schema column adds): - `record_soulsync_library_entry` writes mbid + isrc columns when present in track_info - Deezer source maps to deezer_id column (regression case for source-aware column resolver) - `record_library_history_download` labels `_download_username= 'auto_import'` as "Auto-Import" not "Soulseek" - `record_download_provenance` registers source_service as "auto_import" not "soulseek" # Verification - 8/8 new context-shape tests pass - 6/6 side-effects tests pass (4 new + 2 existing) - 207 imports tests pass - 2359 full suite passes (+12 from baseline 2347, no regressions) - 1 pre-existing flake (`test_watchdog_warns_about_stuck_workers`, passes in isolation, unrelated to this change) - Ruff clean	1 month ago
Broque Thomas	eb68873ec9	WHATS_NEW: keep dev-cycle entries under 2.4.3 (no premature 2.4.4 block) ... Per the semver workflow the version string only bumps at release time, so the running dev work on the 2.4.3 line should stay listed under 2.4.3 (not pre-create a 2.4.4 block). Merged the prior '2.4.4' key's six dev entries into the top of '2.4.3', above the existing "May 8, 2026 — 2.4.3 release" date marker, with a "Unreleased — 2.4.3 patch work" date marker so the visual split between unreleased + released entries is preserved. `_getLatestWhatsNewVersion` resolves to the current build version (2.4.3 in `_SOULSYNC_BASE_VERSION`); with the 2.4.4 key gone, the helper modal now surfaces the dev work alongside the released entries when the user opens "What's New", instead of being silently hidden until a future build bump. The release-time bump remains the canonical step that splits "unreleased" entries off into their own version block — done as the last commit on dev before merging dev → main. No code changes — pure WHATS_NEW reorganisation.	1 month ago
Broque Thomas	8a6ee7a2c7	Auto-import: bounded ThreadPoolExecutor + per-candidate UI state isolation ... # Concurrency model Pre-refactor concurrency was emergent + unbounded: - The worker's `_run` thread called `_scan_cycle` every 60s, processing candidates synchronously in a for-loop. - The `/api/auto-import/scan-now` endpoint spawned a fresh `threading.Thread(target=_scan_cycle)` per click — extra parallel scan cycles on top of the timer. - Multiple "Scan Now" clicks during in-flight processing → multiple threads racing on `_processing_paths` / `_folder_snapshots` state, no upper bound on concurrent scanners. - `stop()` didn't wait for in-flight processing — could leave file moves / tag writes / DB inserts mid-flight. Refactor to the pattern Cin uses elsewhere (`missing_download_executor`, `sync_executor`, `import_singles_executor` all use `ThreadPoolExecutor(max_workers=3, thread_name_prefix=...)`): - **One scan thread** — both timer + manual triggers go through `trigger_scan()`, gated by a non-blocking `_scan_lock`. Duplicate triggers no-op instead of stacking parallel scanners. - **Bounded executor** — `ThreadPoolExecutor` (default 3 workers, configurable via `auto_import.max_workers`) runs per-candidate work. Each candidate runs to completion in its own pool thread; up to N candidates run in parallel. - `_scan_and_submit()` is fast — just enumeration + executor submit, returns immediately, doesn't block on per-candidate work. - `_process_one_candidate(candidate)` holds the per-candidate logic identical to the old for-loop body, lifted into a method so the pool can run multiple instances concurrently. - `_submitted_hashes` set + lock dedupes candidates across the timer + manual triggers so a candidate already queued / running doesn't get re-submitted. - `stop()` calls `executor.shutdown(wait=True)` — clean shutdown, no orphaned file ops. # Per-candidate UI state isolation The executor refactor opened two concurrency holes that the old sequential model masked. Both fixed in this commit: 1. **Scalar UI fields stomped across pool workers.** Pre-refactor `_current_folder` / `_current_status` / `_current_track_*` were safe under the sequential model — only one candidate processed at a time, so the fields tracked the in-flight one. With three pool workers writing the same fields, the polling UI saw garbage like "Processing AlbumA, track 7/14: SongFromAlbumB". Replaced with `_active_imports: Dict[hash, _ActiveImport]` keyed on folder_hash, gated by `_active_lock`. Each pool worker owns its own entry. Helpers `_register_active` / `_update_active` / `_unregister_active` / `_snapshot_active` are the only API. 2. **Stats counters not thread-safe.** `self._stats[k] += 1` is read-modify-write — under load, parallel pool workers drop increments. New `_stats_lock` + `_bump_stat()` helper wraps every mutation. `get_status()` reads under the same lock and returns a copy. # Endpoint change `/api/auto-import/scan-now` no longer spawns its own scan thread — calls `auto_import_worker.trigger_scan()` (which routes through the shared lock + executor). Multiple clicks while a scan is in flight no-op deterministically. Endpoint still wraps the call in a daemon thread so the HTTP response returns immediately even if the staging walk is slow. # Backward compat The scalar `_current_folder` / `_current_status` / `_current_track_*` fields are preserved as **read-only properties** that resolve to the FIRST active import. The existing `get_status()` payload still includes those fields populated from the first entry — single-import UIs (and the test fixture) keep working unchanged. New `active_imports` array exposes the full multi-candidate state for parallel-aware UIs. # Behavior preserved - Per-candidate identify / match / process logic byte-identical - Live-progress state preserved (per candidate now) - Stability gate / already-processed dedup preserved - `_record_in_progress` / `_finalize_result` UI rows preserved - Tag-based loose-file grouping unchanged # Behavior changes - Multiple albums process IN PARALLEL up to `max_workers` - "Scan Now" while scan in progress no-ops (was: spawned another) - `stop()` waits for in-flight pool work via `shutdown(wait=True)` - Auto-import card now lists each in-flight album (one line per active import) instead of a single shared progress line # UI `webui/static/stats-automations.js`: - Progress widget reads `active_imports` array, renders one line per in-flight album with per-candidate status / track index - Falls back to the legacy summary line when payload doesn't carry `active_imports` (older backend) - Per-row "live processing" lookup now matches by `folder_hash` through the array instead of by `folder_name` against scalars # Tests added (`tests/imports/test_auto_import_executor.py`) - Pool config: default max_workers=3, configurable via constructor + via `auto_import.max_workers` config, floors at 1 - Scan lock: 5 concurrent `trigger_scan()` calls run only 1 scan while lock held; releases properly so subsequent triggers run - Executor dispatch: 5 candidates → 5 process calls via the pool - Bounded parallelism: max_workers=3 caps at 3 concurrent; max_workers=2 caps at 2 - Cross-trigger dedup: candidate submitted in scan A doesn't get re-submitted by scan B while still in-flight - Graceful shutdown: `stop()` blocks until in-flight pool work finishes - Per-candidate state isolation: 2 parallel workers updating their own candidate state don't interfere — each candidate's track_index / track_name / folder_name reads back exactly as written for that hash - `get_status()` returns coherent `active_imports` array with one entry per in-flight candidate; aggregate top-level `current_status` is 'processing' when any entry is processing - Unregister removes only that candidate, others stay visible - Stats counter thread-safety: 1000 parallel bumps land at 1000 (the read-modify-write race regresses without the lock) - `get_status()` stats snapshot is a copy, not a live reference # Verification - 17 new tests pass (executor + state isolation) - 2347 full suite passes (1 pre-existing flaky test — `test_watchdog_warns_about_stuck_workers` — passes in isolation, unrelated) - Ruff clean	1 month ago
Broque Thomas	3246490800	Auto-import: MBID/ISRC fast paths + duration sanity gate ... Brings the auto-import matcher to picard / beets / roon parity by reaching for the existing AcoustID-grade infrastructure (typed Album foundation, integrity check thresholds) and layering id-based exact matches on top of the fuzzy scorer. Picard-tagged libraries now land every track with full confidence on the first pass. Three layered phases in `core/imports/album_matching.match_files_to_tracks`: 1. **MBID exact match** — file has `musicbrainz_trackid` tag, source returns the same id → instant pair, full confidence, no fuzzy scoring. Picard's primary identifier; per-recording. 2. **ISRC exact match** — file has `isrc` tag, source returns the same id → same fast-path, slightly lower priority than mbid (isrc can be shared across remasters). Both ids normalised before compare (uppercase + strip dashes/spaces for isrc, lowercase for mbid). 3. **Duration sanity gate** — files in the fuzzy phase whose audio length differs from the candidate track's duration by more than `DURATION_TOLERANCE_MS` (3s, matching the post-download integrity check) are rejected before scoring runs. Defends against the cross-disc / cross-release / wrong-edit problem the integrity check used to catch only AFTER the file had already been moved + tagged + db-inserted. Tag reader (`_read_file_tags`) extended: - Reads `isrc` (uppercased, strip / / spaces normalisation deferred to matcher) - Reads `musicbrainz_trackid` as `mbid` (lowercased) - Reads `audio.info.length` and converts to `duration_ms` to match the metadata-source convention Metadata-source layer (`_build_album_track_entry`) extended: - Propagates `isrc` from top-level OR `external_ids.isrc` (spotify shape — would otherwise be stripped before reaching the matcher) - Propagates `musicbrainz_id` from top-level OR `external_ids.mbid` / `external_ids.musicbrainz` - Without this layer, fast paths would silently never fire in production even though unit tests pass — pinned by `test_album_track_entry_propagates_isrc_and_mbid_from_source` 18 new tests in `tests/imports/test_album_matching_exact_id.py`: - Direct: `find_exact_id_matches` with mbid, isrc, isrc normalisation, mbid > isrc priority, spotify-shape `external_ids.isrc`, no-id empty result, file-used-at-most-once - Direct: `duration_sanity_ok` within / outside tolerance, missing durations defer - End-to-end via `match_files_to_tracks`: mbid match short-circuits fuzzy scoring, id-matched files excluded from fuzzy phase, duration gate rejects wrong-disc collisions in fuzzy phase, normal matches pass through the gate, missing durations fall through, deezer seconds-vs-ms conversion, full picard-tagged 10-track album via mbid only - Production-shape: `_build_album_track_entry` propagates isrc + mbid from spotify-shape (`external_ids.isrc`) AND itunes-shape (top- level `isrc`) Verification: - 35 album-matching tests pass total (17 helper + 18 fast-path) - 23 multi-disc tests still pass after the extension (additive) - Full suite: 2311 passed (+18 new), 1 pre-existing flaky timing test failure (`test_watchdog_warns_about_stuck_workers` — passes in isolation, fails only in full-suite runs, unrelated to this PR) - Ruff clean For users: - Picard / Beets / Mp3Tag-tagged libraries (anyone who's organised their music) get instant perfect-confidence matches every time. - Soulseek-tagged downloads (which usually carry isrc when sourced via metadata-aware soulseekers) get the fast path too. - Naively-named files with no useful tags fall through to the improved fuzzy + duration-gated path — same correctness as before for the common case, much harder for the matcher to confidently pair the wrong file. - One step closer to standalone-DB feature parity with plex / jellyfin / navidrome scanners. Acoustid fingerprint fallback (for files with NO useful tags AND no MBID/ISRC) is the next followup PR.	1 month ago
Broque Thomas	c03edc3cb4	Auto-import: respect disc_number in dedup + match scoring ... Caught while live-testing the #524 fix with kendrick lamar mr morale & the big steppers (3 discs). User dropped discs 1+2 loose in staging root + disc 3 in its own folder, every file perfectly tagged with disc_number/track_number/title — only 9 tracks ended up in the library, the rest got integrity-rejected and quarantined. Two related bugs in `AutoImportWorker._match_tracks`: 1. **Quality dedup keyed on track_number alone.** The dedup loop kept `seen_track_nums[track_number] = file` and dropped any later file with the same number, treating it as a quality duplicate. On a multi-disc release where every disc has tracks 1..N, that collapses the album to one disc's worth of files BEFORE the matcher runs. User's 18 loose disc-1+disc-2 files reduced to 9 before any title/disc info was even consulted. 2. **Match scoring ignored disc_number.** The 30% track-number bonus fired whenever `ft[track_number] == track_num` regardless of disc. File with tag (disc=2, track=6, "Auntie Diaries", 281s) got the full bonus matching API track (disc=1, track=6, "Rich Interlude", 103s) — wrong file → wrong destination → integrity check correctly rejected and quarantined the file. Same for tracks 7, 8, 9. Fix: - Dedup keys on `(disc_number, track_number)` tuples — multi-disc files with parallel numbering all survive. - Match scoring's 30% bonus only when BOTH disc AND track agree. Cross-disc same-track-number collisions get a small 5% consolation bonus so title similarity has to carry the match (covers cases where tag disc info is missing or wrong). - API track disc_number read from `disc_number` (Spotify) / `disk_number` (Deezer) / `discNumber` (iTunes) defaulting to 1. 4 new pinning tests in `tests/imports/test_auto_import_multi_disc_matching.py`: - 18-file 2-disc regression case (dedup preserves all) - (disc=2, track=6) file matches API (disc=2, track=6) track, not the disc-1 same-numbered track - Single-disc albums still match normally (no regression) - Quality dedup within a single (disc, track) position still picks higher-quality format (.flac over .mp3) Verification: - 2268 full pytest suite passes (+4 new), 1 skipped, 0 failed - Ruff clean Same branch as the #524 fix because both surfaced from the same import session — easier reviewer context if they ship together.	1 month ago
Broque Thomas	f58f202d32	Fix manual album import losing source — issue #524 ... radoslav-orlov reported every imported album landing in the soulsync standalone library as "Unknown Artist" + the raw 10-digit album id as the title + 0 tracks. Audit traced it to the click handler in the import page dropping the source-of-the-album_id on its way to the backend match endpoint. Root cause: `importPageSelectAlbum(albumId)` (the onclick on every suggestion / search-result card) only passed the album_id string. The full search response carried `source`, `name`, and `artist` per row — the backend's `get_artist_album_tracks` needs source so it can route the lookup to the metadata source the id actually came from. Without it, the source chain tries each source's `get_album(id)` against an id shaped for a different source — a Deezer numeric id against Spotify's id format returns 404, against iTunes's collectionId range returns 404, etc. — and falls through to the failure-fallback dict in `get_artist_album_tracks`: { 'success': False, 'album': {'name': album_name or album_id, 'total_tracks': 0, 'release_date': '', ...}, # no artist field at all 'tracks': [], } That broken album dict then flowed through `build_album_import_context` → post-processing pipeline → `record_soulsync_library_entry`, writing "Unknown Artist" + album_id-as-title + 0 tracks rows into the soulsync standalone library tables. Why hybrid users hit it most: a Spotify-primary user searching for an album → search returns the Spotify result PLUS Deezer fallbacks (via `_search_albums_for_source`'s priority chain). Clicking a Deezer fallback row then sent only the Deezer id to /album/match without flagging that source — Spotify-first chain failed against the Deezer id and the broken fallback got written. Fix: Frontend (`webui/static/stats-automations.js`): - New `importPageState._albumLookup: { albumId: { id, name, artist, source } }` populated by both card renderers (`_renderSuggestionCard` + the search-results render block) before they emit the onclick. - `importPageSelectAlbum` reads source / name / artist from that cache and includes them in the match POST body, so the backend routes to the correct provider's `get_album` on the very first try. - `_escAttr` applied to album_id in the onclick (defensive — ids shouldn't contain quotes but `_escAttr` was already being used on every other field interpolated into onclick attributes). Backend (`web_server.py:import_album_match`): - Defensive log warning when source is missing from the request body. Catches any future regression where another caller (curl / third-party / new UI flow) drops source again — it'll show up as a visible warning in app.log instead of silently corrupting the library. Verification: - Full pytest suite: 2264 passed, 1 skipped, 0 failed - Ruff clean - JS syntax clean - Manual repro requires a real user flow (search albums on the import page → click one → import) which isn't covered by the existing unit tests; reviewer should verify against issue #524's steps before merge.	1 month ago
Broque Thomas	e20994e1c7	Manual picks: stream results, don't auto-retry, fix stuck-at-0% ... Three follow-on fixes to the manual-search candidates modal once people started actually using it: 1. NDJSON streaming. Manual search waited for every source to return before showing anything. Now streams one event per source as each completes — header line, source_results per source, done terminator. Frontend appends rows incrementally via response.body.getReader(). 2. Manual picks no longer auto-retry on failure. New _user_manual_pick flag set on the task in /download-candidate. Both monitor retry paths (not-in-live-transfers stuck + Errored state) bail on the flag. Surfaces the failure to the user instead of silently picking a different candidate via fresh search. 3. Non-Soulseek manual picks (youtube/tidal/qobuz/hifi/deezer/ soundcloud/lidarr) no longer stuck at "downloading 0%" forever. The live_transfers IF branch now marks manual-pick tasks failed directly when the engine reports Errored, instead of deferring to the monitor (which bails on manual picks). Engine fallback in else branch covers the rare race where the orchestrator's pre-populated transfer lookup is missing the entry. Plus a deadlock fix discovered along the way: the new failure path synchronously called on_download_completed while holding tasks_lock, which itself re-acquires the same Lock — non-reentrant threading.Lock self-deadlocked the polling thread. While wedged, every other endpoint that needed the lock (including /candidates → other failed rows couldn't open modals) hung waiting. Moved completion callbacks onto a daemon thread so the lock releases first. Plus failed/not_found/cancelled rows are now ALWAYS clickable (not just when the auto-search cached candidates) — the modal carries the manual search bar, which is the user's recourse for empty results. Plus manual download worker now runs on a dedicated thread instead of competing with the batch's 3-worker missing_download_executor pool — saturated batches no longer queue manual picks indefinitely. All scoped to manual picks via the _user_manual_pick flag — auto attempt flow byte-identical to before. Engine fallback gated on the flag too so auto attempts in the else branch keep the original do-nothing behavior (safety valve handles the stuck-forever case). Also dropped _handle_failed_download from web_server.py — defined but had no callers (dead code). 17 new unit tests pin the gate behavior: - engine fallback: Errored/Cancelled/Succeeded/InProgress transitions, manual-pick gate, terminal-state skip, soulseek skip, missing download_id skip, engine returning None, orchestrator exception - monitor: manual-pick skips not-in-live-transfers retry + Errored retry - IF-branch end-to-end: Errored marks failed, "Completed, Errored" hits failure branch, auto attempts defer to monitor Manual-search endpoint tests rewritten for NDJSON: 11 cases (validation, single-source dispatch, parallel "all" dispatch, one-event-per-source streaming shape, unconfigured-source skip + reject, header metadata, per-source exception isolation). Full suite 2259 passed, 1 skipped.	1 month ago
Broque Thomas	996575fab3	Add manual search to the failed-track candidates modal ... When an auto-download fails or returns "not found" with leftover candidates, the user can already click the status cell to open a modal showing those candidates and pick a different one. This adds a manual search bar to that modal — type any query, hit search, get a fresh round of results without having to bail out and start over from the main search page. Solves the case where the auto-query was bad (featured artist not in title, parentheticals like "(Remastered 2019)" tripping the matcher, slight artist-name variants, transliteration) but the file genuinely exists on the source. Frontend (downloads.js) - Added a manual-search section above the existing auto-candidates table inside the candidates modal. - Source picker is smart per download mode: - Single-source mode (soulseek-only / youtube-only / etc) shows a "Searching X" label, no dropdown. - Hybrid mode shows a dropdown with "All sources" default + every configured source. Picking "All" runs parallel searches across them and tags each result row with its source badge. - Only configured sources show up; unconfigured are hidden. - Validation: button disabled until query length >= 2, "Type at least 2 characters" hint until threshold crosses. - Loading state on search button while the request is in flight. - Manual results render in a separate table above the existing auto-candidates table, using the same row template (file / quality / size / duration / user / ⬇ button) so the renderer helper is shared. - Click ⬇ reuses the existing `downloadCandidate(taskId, candidate, trackName)` flow — same retry path, same AcoustID verification when the file lands, no shortcut around the safety net. - Re-running the search with a different query replaces the previous manual results. Backend (web_server.py) - Extended `GET /api/downloads/task/<id>/candidates` response with: - `download_mode` (e.g. 'hybrid', 'soulseek') - `available_sources` (list of configured source IDs + labels) - `source` field on each candidate (purely additive — frontend auto-renderer ignores it on legacy code paths, manual-search renderer uses it for the badge) - Added `POST /api/downloads/task/<id>/manual-search`: - Body: `{ query, source: 'all' | <source_id> }` - Validates query length (>=2 trimmed) → 400 - Validates source against the configured-sources gate → 400 (rejects unconfigured sources even when explicitly named) - For 'all': parallel `ThreadPoolExecutor` dispatch across every configured download source, merged results - For specific source: just that source - Returns same shape as `/candidates` so the frontend renderer is reused - New module-level helpers: `_STREAMING_SOURCE_NAMES`, `_infer_candidate_source`, `_serialize_candidate`, `_list_available_download_sources`. The existing `/candidates` endpoint also goes through `_serialize_candidate` so the source badge is consistent across both flows. Behavior preserved - Existing modal layout / candidates table / ⬇ button are byte-identical when the user doesn't use manual search. - `downloadCandidate()` JS function untouched. - `/candidates` and `/download-candidate` endpoints backwards-compatible — only NEW fields added, nothing changed or removed. Tests `tests/test_manual_search_endpoint.py` — 10 tests: - `test_manual_search_validates_query_length` - `test_manual_search_validates_source` (whitelist gate) - `test_manual_search_handles_task_not_found` (404) - `test_manual_search_dispatches_to_configured_source_only` - `test_manual_search_all_dispatches_parallel` - `test_manual_search_skips_unconfigured_sources` - `test_manual_search_rejects_unconfigured_source_explicitly` - `test_manual_search_returns_same_shape_as_candidates` - `test_manual_search_single_source_mode_lists_source` (verifies `available_sources` reflects the active mode) - `test_manual_search_isolates_per_source_exceptions` (one source throwing doesn't kill the merged result) 2242/2242 full suite green (was 2232 + 10 new). Ruff clean. JS parses clean.	1 month ago
Broque Thomas	d556ec0fa7	Bump version to 2.4.3 + make sidebar version dynamic ... - `_SOULSYNC_BASE_VERSION` 2.4.2 → 2.4.3 - helper.js — flip 2.4.3 WHATS_NEW header to "May 8, 2026 — 2.4.3 release"; bump fallback default from 2.4.2 → 2.4.3 - docker-publish.yml — manual-trigger default tag 2.4.2 → 2.4.3 Drive-by — make sidebar version + version-modal subtitle dynamic. The sidebar version button (`v2.4.1`) and version-modal subtitle (`Version 2.4.1 — Latest Changes`) were hardcoded text in the HTML. 2.4.2 shipped without these getting bumped — silent drift, easy to miss at every release. Added a Flask context_processor that injects `soulsync_version` and `soulsync_base_version` into every template, then templated the two hardcoded values: v{{ soulsync_base_version }} Version {{ soulsync_base_version }} — Latest Changes Now bumping `_SOULSYNC_BASE_VERSION` updates the UI everywhere it's rendered. No more "I forgot to bump the sidebar" at release. 2232/2232 full suite green. Ruff clean. JS parses clean.	1 month ago
Broque Thomas	d75ae48981	Discover: sharpen track selection (diversity, source-aware popularity, library dedup, SQL genre) ... Four selection-quality fixes on the SoulSync-made discover playlists. None change public method signatures; all are tightenings on what's already there. (1) Diversity for Hidden Gems + Discovery Shuffle Both used to be `RANDOM() LIMIT N` with no diversity. Could return 50 tracks from one artist or 20 from one album if the discovery pool happened to be skewed. Both now over-fetch 3x and run the existing `_apply_diversity_filter`: - Hidden Gems: max 2 per album, 3 per artist - Discovery Shuffle: max 2 per album, 2 per artist (tighter — shuffle should feel maximally varied) (2) Source-aware popularity thresholds `popularity >= 60` for "Popular Picks" and `popularity < 40` for "Hidden Gems" was Spotify-shaped (0-100 scale). Deezer writes its `rank` value into that column (often six-digit integers); iTunes writes nothing meaningful. For Deezer-primary users: - Popular Picks pulled essentially everything (rank >= 60 = all) - Hidden Gems pulled essentially nothing (rank < 40 = none) New `_get_popularity_thresholds(source)` helper returns per-source values: - Spotify: (60, 40) — the existing 0-100 scale - Deezer: (500_000, 100_000) — ballpark from real rank values - iTunes / unknown: (None, None) — skip the popularity filter entirely, fall back to random + diversity `get_popular_picks` and `get_hidden_gems` now consult the helper. When threshold is None they skip the popularity SQL filter. Diversity + ID gate still apply. (3) Push genre keyword filter into SQL `get_genre_playlist` used to fetch `limit=1_000_000` rows into Python then run a substring keyword filter on `artist_genres`. Bad on big discovery pools. Now the keyword OR chain is generated as SQL placeholders: AND (artist_genres LIKE ? OR artist_genres LIKE ? OR ...) Each placeholder gets `f'%{keyword.lower()}%'` via `extra_params`. `fetch_limit` drops back to `limit * 10`. `_genre_matches` Python helper deleted (only intra-file caller; verified via grep). Parent-genre expansion via `GENRE_MAPPING` preserved — keywords list feeds the LIKE chain unchanged. (4) Filter out tracks already in library Discovery pool can include tracks the user already owns. Hidden Gems / Shuffle / Popular Picks shouldn't surface those. `_select_discovery_tracks` gained `exclude_owned: bool = True` parameter. When True, adds a correlated NOT EXISTS subquery against the `tracks` table covering all 3 source IDs: AND NOT EXISTS ( SELECT 1 FROM tracks t WHERE (t.spotify_track_id IS NOT NULL AND t.spotify_track_id = discovery_pool.spotify_track_id) OR (t.itunes_track_id IS NOT NULL AND t.itunes_track_id = discovery_pool.itunes_track_id) OR (t.deezer_id IS NOT NULL AND t.deezer_id = discovery_pool.deezer_track_id) ) Note column-name asymmetry: tracks.deezer_id vs discovery_pool.deezer_track_id. Inline comment marks the trap. All 5 public discovery methods automatically benefit (default True). Seasonal Playlist doesn't go through the helper so it's unaffected (curated content, dedup is wrong intent there). Tests 12 new tests in `tests/test_personalized_playlists_id_gate.py` (27 total in the file): - Hidden Gems + Discovery Shuffle apply diversity (cap proven by inserting 10 same-artist + same-album rows and asserting return count ≤ per-album cap) - Popularity thresholds: Spotify (60, 40), Deezer larger scale, iTunes None / None - Popular Picks skips threshold filter when None - Genre playlist pushes filter to SQL (parent + child genre expansion) - Owned-track exclusion: filtered when match, kept when no match, opt-out flag works - Deezer column-name asymmetry pinned (regression footgun) Test fixture re-added the minimal `tracks` table (4 columns: id, spotify_track_id, itunes_track_id, deezer_id) — only what the new NOT EXISTS subquery needs to join. Plus `insert_library_track` helper. Verification - 27/27 in this test file pass (15 prior + 12 new) - 2232/2232 full suite green - ruff clean LOC delta: - core/personalized_playlists.py: 1030 → 1101 (+71) - tests/test_personalized_playlists_id_gate.py: 352 → 616 (+264)	1 month ago
Broque Thomas	959562f6b0	Delete Recently Added / Top Tracks / Forgotten Favorites / Familiar Favorites ... Owner decision: not worth shipping. The four library-driven personalized sections were stubbed returning [] for ages because their schema prereqs didn't exist; the prior commit re-enabled them by routing through a new `_select_library_tracks` helper. Owner reviewed and chose to delete the sections entirely instead. Removed everywhere: - `core/personalized_playlists.py` — `get_recently_added`, `get_top_tracks`, `get_forgotten_favorites`, `get_familiar_favorites` + the `_select_library_tracks` helper (no other callers; verified via grep). - `web_server.py` — 4 route handlers (`/api/discover/personalized/recently-added`, `top-tracks`, `forgotten-favorites`, `familiar-favorites`). - `webui/index.html` — 4 `<div class="discover-section">` blocks (`#personalized-recently-added`, `#personalized-top-tracks`, `#personalized-forgotten-favorites`, `#personalized-familiar-favorites`). - `webui/static/discover.js` — 4 load functions (`loadPersonalizedRecentlyAdded`, `loadPersonalizedTopTracks`, `loadPersonalizedForgottenFavorites`, `loadFamiliarFavorites`), plus their entries in `loadDiscoverPage`'s Promise.all, plus 4 module-level state vars + 6 dead branches across `openDownloadModalForDiscoverPlaylist` / `startDiscoverPlaylistSync` and the sync-progress / rehydrate dispatchers. - `webui/static/helper.js` — 4 tooltip / docs entries. - `webui/static/sync-spotify.js` — 1 stale rehydrate dispatcher branch (`discover_familiar_favorites`) caught during the global grep pass. - `tests/test_personalized_playlists_id_gate.py` — 3 library-method tests + the test infrastructure that supported them (`tracks` schema, `insert_library_track` helper). Documentation header updated to reflect the deletion. Net: -527 / +2 lines across 7 files. What stays: - Daily Mixes (also in personalized package, intentionally paused — separate decision). - Popular Picks + Hidden Gems + Discovery Shuffle (alive, not affected by this deletion). - All 14 tests in the personalized-playlists test file still pass. - The PersonalizedPlaylistsService lift from the prior commit (`_select_discovery_tracks` etc) — those are still in active use by the surviving discovery_pool methods. DISCOVER_TRACK_SELECTION_REVIEW.md at repo root contains historical references to the four deleted endpoints. Treated as historical context (same policy as WHATS_NEW), left alone. 2219/2219 full suite green (was 2222 - 3 deleted tests = 2219). JS parses clean, ruff clean.	1 month ago
Broque Thomas	44dd7f980f	Discover: unify Decade + Genre tabbed browsers ... Both tabbed-browser sections — Time Machine ("Decade") and Browse by Genre — re-implemented the same lifecycle by hand: fetch tabs list, render the tab strip, attach click handlers, fetch content per tab, render track list with sync + download action buttons + sync-status block, handle empty/error/loading states. ~314 lines of identical boilerplate split across two browsers. Lifted into one shared `createTabbedBrowserSection(config)` helper. Each browser is now a thin wrapper: ```js const ctrl = createTabbedBrowserSection({ id: 'decade-browser', tabsContainerEl: '#decade-tabs', contentContainerEl: '#decade-content', fetchTabs: async () => { ... }, renderTabButton: (tab, isActive) => `<button>...</button>`, fetchTabContent: async (tab) => { ... }, renderTabContent: (tracks, tab) => `...`, onTabContentRendered: (tab, contentEl) => { ... }, emptyMessage / errorMessage, }); ``` Migrated: - `loadDecadeBrowserTabs` 85 → 3 lines - `loadDecadeTracks` 67 → 3 lines - `loadGenreBrowserTabs` 92 → 3 lines - `loadGenreTracks` 70 → 3 lines Helper: ~125 lines + ~100 lines of per-browser config blocks + ~25 lines of shared `_renderTabbedTrackList` (the two browsers had byte-identical track-row markup so it lifted cleanly). Public function names preserved — the four migrated functions stay on the same signature so existing callers (`loadDiscoverPage`, refresh buttons, inline handlers) don't change. Side effects preserved — `decadeTracksCache[year]`, `activeDecade`, `genreTracksCache[name]`, `activeGenre`, `availableGenres` still mutated at the same lifecycle moments. The decade-specific `startDecadeSync(decade)` and genre-specific `startGenreSync(name)` sync-button handlers stay where they are; they're click handlers attached to rendered content, not part of the tab lifecycle. What didn't fit (intentionally left alone): - `_renderCompactTrackRow` (the existing shared track-row helper) is NOT used by the tabbed browsers — they had their own template with a `track_data_json` fallback chain `_renderCompactTrackRow` doesn't do. Unifying these two would change behavior for non-tabbed sections, so the tabbed-browser variant lives as `_renderTabbedTrackList`. Future cleanup could merge them by giving `_renderCompactTrackRow` an opt-in fallback flag. - `switchDecadeTab` / `switchGenreTab` still know about cache shape so they can skip refetch on already-loaded tabs. Keeping that in the per-browser switch is fine — it's a click handler, not lifecycle. Net: 8546 → 8578 LOC on `discover.js` (+32). Helper boilerplate offsets the line count, but the win is single-source-of-truth, not raw line reduction. `node --check` clean. 2222/2222 full suite green.	1 month ago
Broque Thomas	c557d9196e	Discover controller — Cin pre-review polish ... Three changes tightening the controller before opening the PR. DROP MAGIC `extractItems` DEFAULTS Controller used to auto-pull `data.items` / `data.albums` / `data.artists` / `data.tracks` / `data.results` when no extractor was supplied. Removed the fallback chain — every section now MUST provide an explicit `extractItems(data) => array`. Validated at register-time so misuse fails immediately, not silently on first load against an endpoint that happened to return two arrays. Cin standard: explicit > implicit. Magic key-grabbing could pick the wrong one in edge cases (e.g. an endpoint returning both `data.albums` and `data.results` would have grabbed albums when the section actually wanted results). All 10 existing controller call sites already passed explicit extractors, so no migration churn — this is purely tightening the contract for future sections. REPLACE `renderItems` NULL-RETURN CONVENTION WITH `manualDom: true` Your Albums and similar sections that delegate to existing renderers that target a CHILD element of `contentEl` used to signal "leave the container alone" by returning null/undefined from `renderItems`. That convention is easy to confuse with an accidental missing-return error. Replaced with an explicit `manualDom: true` config flag. Renderer is still called for its side-effects, controller just skips the innerHTML swap. Clearer intent at the call site. Updated `loadYourAlbums` to use the new flag. PIN THE CONTROLLER CONTRACT WITH JS TESTS Added `tests/static/test_discover_section_controller.mjs` — 32 tests covering the controller's lifecycle contract: - Config validation (every required field, mutual exclusivity of fetchUrl/data, type checks on contentEl) - Happy-path fetch → parse → render - Empty state (default empty render, hideWhenEmpty + sectionEl, success=false treated as empty, custom isSuccess override) - Stale state (fires when isStale returns true, wins over empty, custom renderStale override) - Error state (HTTP non-ok, fetch throws, showErrorToast fires window.showToast, default off doesn't fire) - No-fetch `data:` mode (value + function form, doesn't call fetch) - manualDom mode (skips innerHTML swap, still calls renderer) - Callable `fetchUrl` (resolved at load time, refresh re-resolves) - Load coalescing (concurrent loads share one fetch) - Refresh bypasses coalescing (re-fires fetch every call) - Hook error containment (throwing renderer/onSuccess hooks don't crash the controller) Runs via Node's stable built-in `--test` runner — no package.json, no jest/vitest dependency, no compile step. Just `node --test`. Pytest wrapper at `tests/test_discover_section_controller_js.py` shells out to node and asserts clean exit, so the JS tests fail the regular pytest sweep if the controller contract drifts. Skipped gracefully when node isn't available or is < 22. Closes the "controller is a contract, pin it at the test boundary" gap that Cin would have flagged on review. VERIFICATION - 2205/2205 full pytest suite green (was 2204 + 1 new wrapper) - 32/32 `node --test` pass on the controller test file directly - ruff clean - node --check clean on all touched JS files	1 month ago
Broque Thomas	dc2323cde6	Discover cleanup: controller extensions, toast errors, migrate skipped sections ... Follow-up to the controller migration commits. Closes out the extension list the per-section migrations surfaced as needed. CONTROLLER EXTENSIONS - Callable `fetchUrl: () => string` — resolves the seasonal-playlist recreate-on-key-change hack from the prior commit. - No-fetch `data:` mode — value or `() => value`. Lets render-only sections like Seasonal Albums use the controller without inventing a fake endpoint. Mutually exclusive with `fetchUrl`; validated up front so misuse fails at register-time. - `beforeLoad(ctx)` hook — runs before the spinner shows. Lets dynamically-inserted sections like Because You Listen To ensure their `contentEl` exists before the visibility check. - `onSuccess(data, ctx)` hook — runs after the success gate but before isEmpty / isStale. Cleaner home for sibling header / subtitle / button updates than folding them into renderItems. - `isStale(items, data)` + `onStale(ctx)` + `renderStale(items, data)` + `staleMessage` — third render state for "data is empty BUT upstream is still discovering". Stale wins over empty when both apply. Default stale UI is the same spinner block used elsewhere. - `showErrorToast: true` config — opens a global `showToast(...)` in addition to the in-section error block. Default off; sections that have no recovery action shouldn't shout at the user. - `renderItems` returning null/undefined now leaves contentEl untouched. Lets a renderer do its own DOM manipulation (e.g. delegating to an existing grid-render fn that targets a child element) without fighting the controller's innerHTML swap. MIGRATED THE 2 SKIPPED SECTIONS - `loadYourAlbums` — uses `isStale`/`onStale`/`renderStale` for the stale-fetch state, `onSuccess` for the subtitle/filters/download side-effects, `hideWhenEmpty` + `sectionEl` for the truly-empty case, `renderItems` returning null since it delegates to the existing `_renderYourAlbumsGrid` + `_renderYourAlbumsPagination`. - `loadSeasonalAlbums` — uses no-fetch `data:` mode because the parent `loadSeasonalContent` already fetched the season payload. `beforeLoad` updates the sibling title/subtitle text. ERROR TOASTS ON ALL MIGRATED SECTIONS Every migrated section now has `showErrorToast: true`. Section load failures surface a global toast instead of silently spinning forever or swallowing into console.debug. Same pattern JohnBaumb #369 asked for at the Python layer, applied at the UI layer. SHARED SYNC-STATUS BLOCK Lifted the duplicated decade-tab + genre-tab sync-status HTML (✓ completed | ⏳ pending | ✗ failed | percentage) into a single `_renderSyncStatusBlock(idPrefix)` helper. Two call sites now share one implementation. ListenBrainz playlists keep their own block because the semantics differ — matching progress (total / matched / failed) vs download progress. DEAD-SECTION AUDIT — NONE DEAD Audited the 13 supposedly-dead hidden sections from DISCOVER_REVIEW.md. All 13 are alive: gated on user data (discovery pool, library content, metadata cache) and self-surface when their data exists via `style.display = 'block'` on the success path. The review's grep missed the toggle. No deletions made. DAILY MIXES ORPHAN CALL Removed the orphaned `loadPersonalizedDailyMixes()` call from `blockDiscoveryArtist` — Daily Mixes is intentionally paused (its load call in `loadDiscoverPage` is commented out) so refreshing it from the post-block hook was a no-op. 2204/2204 full suite green. JS parses clean (`node --check`).	1 month ago
Broque Thomas	4ee78bb973	Migrate 7 more discover sections to the shared controller ... Follow-up to the foundation commit. Drops the hand-rolled try/catch + spinner injection + empty-state HTML + error-swallow in seven sections by routing them through `createDiscoverSectionController`. Each section keeps its existing public function name + signature so callers, refresh buttons, and dashboard wiring don't notice the swap. Migrated: - `loadDiscoverReleaseRadar` (Fresh Tape) - `loadDiscoverWeekly` (The Archives) - `loadDecadeBrowser` (Time Machine intro carousel) - `loadGenreBrowser` (Browse by Genre intro carousel) - `loadSeasonalPlaylist` (Seasonal Mix) - `loadYourArtists` - `loadBecauseYouListenTo` Skipped (don't fit the controller's single-fetch / single-render-target shape): - `loadYourAlbums` — paginated grid + filters, updates four separate UI elements (subtitle, filter chips, download button, grid). - `loadSeasonalAlbums` — receives pre-fetched data from `loadSeasonalContent`; no fetch URL to satisfy. Hidden / dead sections (~13 of them — `loadPersonalized*`, `loadDiscoveryShuffle`, `loadFamiliarFavorites`, `loadCache*`) untouched in this pass. Separate audit commit will surface or kill them. Two side-effects worth noting: - `loadDecadeBrowser` and `loadGenreBrowser` migrated for completeness, but neither appears wired into `loadDiscoverPage` or any inline handler. May be dead code — flagged for the audit pass. - `loadSeasonalPlaylist` needs a per-load fetch URL (varies by `currentSeasonKey`); worked around by recreating the controller when the key changes. Cleaner option: extend the controller to accept a `fetchUrl: () => string` callable form. Tracked in the follow-up extension list below. Controller extension candidates surfaced for follow-up: - Callable `fetchUrl` (resolves the seasonal playlist recreate-on-key-change hack) - Explicit `isStale` / `onStale` hook (so Your Artists doesn't fold stale handling into renderItems) - `beforeLoad` / `ensureContentEl` hook (so Because You Listen To can let the controller own the dynamic container creation) - No-fetch `data:` mode (so render-only sections like Seasonal Albums can use the controller too) - `onSuccess(data)` hook (cleaner home for header / subtitle side-effects vs folding them into renderItems) Net: -76 lines in `discover.js` even after adding the per-section render helpers. 2204/2204 full suite green. JS parses clean.	1 month ago
Broque Thomas	07a71f0432	Discover section controller foundation + migrate Recent Releases ... Every section on the discover page (Recent Releases, Your Artists, Your Albums, Seasonal Albums, Seasonal Mix, Fresh Tape, The Archives, Build Playlist, Time Machine, Browse by Genre, ListenBrainz Playlists, Because You Listen To, plus ~13 hidden sections) currently re-implements the same lifecycle by hand: 1. show a loading spinner in the carousel container 2. fetch the section's endpoint 3. parse the response, decide if the data is empty 4. either render the items, show an empty-state, or show an error 5. wire post-render handlers (download buttons, hover behavior, etc) 6. maybe expose refresh() ~30 sections worth of duplicated boilerplate, all subtly drifting. Different empty-state messages. Different error handling (some `console.debug`, some silently swallowed, some leave the spinner spinning forever). Different sync-status icons (✓/⏳/✗ vs ♪/✓/✗). No consistent error toast. Lifted the lifecycle into a shared `createDiscoverSectionController` in `webui/static/discover-section-controller.js`. Renderers stay per-section because section data shapes legitimately differ — album cards vs artist circles vs playlist tiles vs track rows. The controller is the wrapper, not a forced visual abstraction. Foundation contract: createDiscoverSectionController({ id: 'recent-releases', // for diagnostic logging contentEl: '#carousel', // selector or Element fetchUrl: '/api/discover/...', extractItems: (data) => [...], // pull list from response renderItems: (items, data, ctx) => '<html>', onRendered: (ctx) => { ... }, // optional post-render hook loadingMessage / emptyMessage / errorMessage: copy sectionEl + hideWhenEmpty: optional whole-section visibility isSuccess / isEmpty: optional gate overrides }) Returns `{ load, refresh, destroy, getState }`. Validates config up front so misuse fails at register-time, not silently on load. Coalesces concurrent loads (same in-flight promise returned) so a double-click or repeated trigger doesn't double-fetch. `refresh()` bypasses the coalesce so the refresh button always re-fires. Errors are logged (console.debug by default, console.error when verboseErrors=true). Renderer hook errors are caught + logged so a buggy render callback can't tear down the controller — keeps the page resilient. Migrated `Recent Releases` as the proof — simplest album-card shape, no source-gating, no refresh button. Verified the contract covers it end-to-end. The legacy `loadDiscoverRecentReleases` entry-point stays public so existing callers don't change; internally it lazy-builds the controller and triggers `load()`. NOT in this commit: - Other section migrations (one section per follow-up commit, keeps reviews small + lets us sequence the work) - Registry-driven section list (so the dead-section audit becomes registry deletions instead of section-by-section removal) - Global error toast wrapper - Per-section "requires X primary source" gate - Sync-status icon renderer unification Once every section is on the controller, the discover-page cleanup work (kill the 13 dead sections, standardize sync-status icons, add error toasts) becomes single-line registry-level edits instead of 30 separate section-by-section rewrites. 2204/2204 full suite green. JS parses clean (`node --check`). Manual smoke deferred until follow-up commits — Recent Releases unchanged on the wire (same endpoint, same payload shape, same render output).	1 month ago
Broque Thomas	6aafcaae93	Bump version to 2.4.2 ... - `web_server.py` — `_SOULSYNC_BASE_VERSION` 2.4.1 → 2.4.2 - `webui/static/helper.js` — flip the 2.4.2 WHATS_NEW header from "Unreleased — 2.4.2 dev cycle" to "May 7, 2026 — 2.4.2 release" so the per-version block stops being filtered out by `_getLatestWhatsNewVersion`. Also bumps the safety-net default inside that helper from 2.4.1 → 2.4.2. - `.github/workflows/docker-publish.yml` — manual-trigger default tag bumped to match. Drive-by fix: escaped a stray single quote in the `Internal: Download Engine` 2.4.2 entry that broke `node --check` on the file (`orchestrator.client('soulseek')` inside a single-quoted desc string silently terminated the string mid-entry). Pre-existing, unrelated to the bump but caught while validating JS parse for the release. VERSION_MODAL_SECTIONS not rotated in this commit — separate editorial pass.	1 month ago
Broque Thomas	1a2da016e4	Add download buttons + bulk action to artist top-tracks sidebar ... Closes #513 (s66jones). The artist detail page already showed a "Popular on Last.fm" sidebar — list of an artist's top tracks by playcount, with a play button per row but no download action. Issue #513 wanted a way to grab those tracks the same way zotify let users grab "top X songs" without pulling the full discography. Pulls from the configured primary metadata source (Spotify `artist_top_tracks`, Deezer `/artist/{id}/top`) when available, falls back to the existing Last.fm display-only mode for sources that don't expose popularity ranking (iTunes / Discogs / MusicBrainz). Source label in the section title shifts to match. Each row gets a hover-revealed download button that wishlists the single track via the existing /api/add-album-to-wishlist endpoint (preserves the track's real album metadata, so the wishlist worker later places the file in its proper album folder). A "Download All" footer button opens the standard download modal in PLAYLIST context, not album context — the virtual playlist_id is `top_tracks_<source>_<artistId>` which doesn't match any of the album-prefix checks in `startMissingTracksProcess` (downloads.js). That keeps `is_album_download=false`, so the master worker doesn't inject a wrapper context as `_explicit_album_context`. Each track downloads using its own real album metadata, files land in proper per-album folders on disk (not a fake "Top Tracks" folder). Backend additions: - `SpotifyClient.get_artist_top_tracks(artist_id, country, limit)` — wraps `spotipy.artist_top_tracks`, returns up to 10 tracks for the market (Spotify's API cap). UI-side limit trim only. - `DeezerClient.get_artist_top_tracks(artist_id, limit)` — wraps `/artist/{id}/top?limit=N`, converts Deezer's raw shape to the same Spotify-compatible dict layout (id, name, artists, album with album_type / total_tracks / images, duration_ms, track_number, disc_number) so downstream code doesn't branch on source. - `GET /api/artist/<id>/top-tracks` — dispatches to whichever client matches the primary source. Resolves per-source artist IDs from the DB row first (matching what /discography already does) so a Spotify ID in the URL still works when Deezer is primary, and vice versa. Returns `{success, source, tracks, resolved_artist_id}` on hit; `{success: False, reason: 'unsupported_source' | 'spotify_not_authenticated' | 'deezer_unavailable' | 'no_tracks_found'}` on miss so the frontend can decide whether to fall through to Last.fm. Frontend: - `_loadArtistTopTracks` tries the metadata source first, falls through to the legacy `/api/artist/0/lastfm-top-tracks` call if the source can't deliver. Section title and per-row UI shift based on which source answered. - New per-row `.hero-top-track-download` button (hover-revealed). - New `.hero-top-tracks-download-all` footer button — only visible when metadata-source mode rendered the list (Last.fm fallback hides it since rows have no track IDs to download). Tests: 10 new tests pin the client methods — - Spotify: returns track list, honors UI limit cap, returns empty when unauthed / artist_id missing / API throws. - Deezer: shape conversion to Spotify-compatible dict, empty when no data / artist_id missing, limit clamping at upper bound, default fallback when limit=0, malformed entries skipped. The Flask endpoint dispatcher itself isn't covered by the new test file because importing web_server at test-collection time spins up worker threads that race with caplog-using tests elsewhere in the suite (specifically test_library_reorganize_orchestrator). Endpoint verified manually; the underlying client methods (the load-bearing logic) are covered. 2204/2204 full suite green (was 2194 + 10 new).	1 month ago
Broque Thomas	dd48dc8c6e	Update style.css	1 month ago
Broque Thomas	01c528fd5f	Reject AcoustID matches whose version disagrees with the expected track ... Discord report (corruption [BWC]): downloads coming through as the instrumental cut when a vocal track was requested. The verification step's `_normalize` function strips parentheticals and version-suffix tags ("(Instrumental)", "- Live", etc) so legitimate name variations don't false-fail the title-similarity check. That also means "In My Feelings" and "In My Feelings (Instrumental)" both normalize to "in my feelings", title similarity is 1.0, and the wrong cut passes verification. Detect the version label on each side BEFORE normalization runs. If the expected and matched recordings disagree on version (one is original, the other is instrumental / live / acoustic / remix / etc), return FAIL — the fingerprint identified a real song, just not the version the caller asked for. Reuses `MusicMatchingEngine.detect_version_type` so the same regex patterns the pre-download Soulseek matcher applies also drive post-download verification. No duplicated tables. Also gates the secondary fallback scan, so a wrong-version variant sitting in the same fingerprint cluster can't win the loop after the best match has already been version-rejected. 6 tests pin the behavior: - instrumental returned for vocal request → FAIL - vocal returned for instrumental request → FAIL - live vs acoustic → FAIL - matching versions on both sides → PASS - original-to-original happy path → PASS (regression guard) - secondary scan skips wrong-version recordings → not PASS 2194/2194 full suite green (was 2188 + 6 new).	1 month ago
Broque Thomas	caa1c198e5	Fix non-admin profiles defaulting to Spotify on search picker ... Closes #515 (jaruca). Search-picker controller in shared-helpers.js resolved the user's configured primary metadata source by fetching `/api/settings`. That endpoint is `@admin_only` (it returns full config including credentials), so non-admin profiles got a 403 and the controller silently fell back to the hardcoded `'spotify'` default — admin's chosen source (deezer / itunes / discogs / etc) was ignored on every non-admin profile, forcing manual reselection each session. Switched to `/status`, which is public and already exposes the resolved `metadata_source` for the dashboard. Same value the picker needs — different endpoint that doesn't gate non-admins. Admins see no behavior change. Non-admins now see admin's configured primary source as the default active icon. Refs #515	1 month ago
Broque Thomas	9602d1827c	Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites ... Catches the silent excepts the awk-based earlier sweeps missed: - Bare `except:` followed by `pass` (also swallows KeyboardInterrupt and SystemExit — actively wrong). Upgraded to `except Exception as e: logger.debug("...: %s", e)`. ~14 sites across connection_detect, soulseek_client, listenbrainz_manager, watchlist_scanner, youtube_client, navidrome_client, jellyfin_client, web_server. - `except Exception:` + pass that the awk pattern missed (e.g. multi-line or unusual whitespace). ~31 sites across automation_engine, database_update_worker, music_database, spotify_client, web_server, others. - 14 legitimate cleanup sites left silent with explicit `# noqa: S110` + comment explaining why (atexit handlers, finally-block conn.close calls). Logging during shutdown can itself crash because file handles get torn down before the handler fires. Also enables `S110` rule in pyproject.toml so this pattern fails CI going forward — drift fails at PR review instead of at runtime against a wedged worker thread. Tests path keeps S110 ignored (test fixtures legitimately use try-except-pass for cleanup). Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep. Verified: `python -m ruff check .` → All checks passed. Verified: `python -m pytest tests/` → 2188 passed. Closes #369	1 month ago
Broque Thomas	4c11375930	Repair job card badge — show pending count, not last-scan count ... Discord report: Duplicate Detector card said "372 findings" and Cover Art Filler said "60 findings", but clicking the Findings tab's Pending filter showed 0. User read it as "findings aren't being created" — looked like a detector bug. Actual cause: the badge sourced ``last_run.findings_created`` (historical "found in last scan") without considering current state. After the user (or bulk-fix automation) resolved or dismissed those findings, they no longer appeared on the Pending tab — but the badge kept showing the last-scan number in red urgent styling. Backend was correct end-to-end: detectors create pending rows, bulk-fix moves them to resolved, Findings tab filters by status. Only the badge display lied about current state. Fix: - ``RepairWorker._get_pending_count_by_job()`` — single SQL aggregation returning ``{job_id: pending_count}`` for every job with pending findings. O(1) lookup per job instead of N round trips. - ``get_all_job_info()`` calls it once per request and adds ``pending_findings_count`` to each job's API response. - ``enrichment.js`` job card now branches on the count: - ``> 0`` → red ``"X pending"`` badge (urgent, action needed) - ``= 0`` AND last scan found something → muted grey ``"X found in last scan"`` (historical context, no action needed) - New CSS class ``.repair-flow-badge.findings-historical`` for the muted slate color so the two states are visually distinct. User-visible result with the screenshotted state (372 dup / 60 cover- art findings, all resolved): - Before: red "372 findings" / "60 findings" — implied 432 things to do, but Findings tab showed 0 pending - After: grey "372 found in last scan" / "60 found in last scan" — the badge text tells the user the count is historical, no surprise when Pending is empty Tests: 3 new tests in ``tests/test_create_finding_dedup_counter.py`` pin the per-job pending count helper: - returns ``{job_id: count}`` based on status='pending' rows only; resolved + dismissed rows excluded - empty dict when no pending findings exist - gracefully returns ``{}`` on DB error (badge falls back to historical count via the existing JS ``or 0`` safety) 2188/2188 full suite green. Pure UI/state-display fix — no detector logic, no backend behavior change.	1 month ago
Broque Thomas	5c69b853b4	Bound slskd HTTP timeout — fixes worker thread deadlock ... GitHub issue #499 (@bafoed). Big initial sync of Spotify playlists worked for 2-3 hours then downloads silently stopped: - 3 active tasks stuck in "Searching" state, replaced every ~10 min by different ones - slskd UI showed no actual searches happening - Debug log: orphaned-task count grew over time, no jobs executed - Container restart was the only fix (bought another 2-3 hours) - Not a rate limit (rates showed 0/min) Root cause: ``core/soulseek_client.py`` constructed ``aiohttp.ClientSession()`` with no timeout at four sites. When slskd hung on a request (overloaded, transient network blip, internal stall), the HTTP call blocked indefinitely — and the worker thread blocked with it. The download executor only has ``ThreadPoolExecutor(max_workers=3)``, so once 3 worker threads were wedged on hung calls, no further downloads could start. Batch-level "stuck detection" (10-minute timer in ``check_batch_completion_v2``) was correctly marking tasks ``not_found`` and trying to start replacements, but the executor pool was exhausted — replacements queued forever inside the executor with no thread to run them. Symptom: tasks rotating every ~10 min at the batch level while the underlying executor stayed wedged. Fix: bounded ``aiohttp.ClientTimeout`` (total 120s, connect 15s, sock_read 60s) on every slskd ``ClientSession`` construction. Module- level constant ``_SLSKD_DEFAULT_TIMEOUT`` so the four sites stay in lockstep — future sites get the same protection by reusing the constant. Why these timeouts are safe: - Every slskd API call is metadata-level (search submission, status polls, download enqueue, transfer state queries). None stream files — slskd handles file transfer via its own peer-to-peer infrastructure entirely outside our HTTP requests. - Legitimate metadata calls finish in seconds. 120s ceiling is ~50× the normal latency. Timeout handling: - ``asyncio.TimeoutError`` caught explicitly BEFORE the generic ``except Exception`` — surfaces "slskd timed out" specifically in logs (debuggable instead of buried as "Error making API request"). - Returns None to the caller (same code path as a 5xx response or any other failure). No new error path; callers already handle None as "request failed". - Worker thread unblocks immediately → executor pool stays healthy → downloads keep flowing. Sites updated: - ``_make_request`` (general /api/v0/ helper, line 152) — used for every slskd API operation - ``_make_direct_request`` (non-/api/v0/ helper, line 235) - ``_explore_api_endpoints`` Swagger fetch (line 1566) — diagnostic - ``_explore_api_endpoints`` per-endpoint probe (line 1617) — diagnostic Tests: 3 new tests in ``tests/downloads/test_soulseek_pinning.py`` pin: - ``_SLSKD_DEFAULT_TIMEOUT`` is bounded (total set, ≤300s ceiling, connect ≤60s) — guards against future regressions that drop or unbound the timeout - ``_make_request`` returns None on ``asyncio.TimeoutError`` rather than raising — pins the caller contract - ``_make_direct_request`` returns None on ``asyncio.TimeoutError`` 2185/2185 full suite green. Closes #499.	1 month ago
Broque Thomas	ca5c93162c	Rewrite Library Reorganize job to delegate to per-album planner ... GitHub issue #500 (@bafoed). Library Reorganize repair job moved album tracks to single-template paths because of a fragile classification heuristic. Concrete symptom: a track at ``Surf Curse/Surf Curse - Nothing Yet (2017)/01 - Christine F.flac`` got proposed for a move to ``Surf Curse/Surf Curse - Christine F/Surf Curse - Christine F.flac`` (single template) instead of staying under the album folder. Root cause: the job had its own tag-reading + transfer-folder-walk + template-application implementation. The classification was ``is_album = (group_size > 1)`` where ``group_size`` was the count of same-album tracks currently sitting in the transfer folder being scanned. Two failure modes: - only one track of an album was in the transfer folder (rest already moved to the library, or not yet downloaded), or - album tags varied slightly across tracks (e.g. ``"Buds"`` vs ``"Buds (Bonus)"``) Either case gave a 1-element group → routed through the SINGLE template → wrong destination. Rewrite — delegate to the per-album planner the artist-detail "Reorganize" modal already uses: - ``core.library_reorganize.preview_album_reorganize`` for path computation (DB-driven, knows the album has N tracks regardless of how many sit in transfer; album-vs-single is structurally correct) - ``core.reorganize_queue.enqueue_many`` for apply mode; the queue worker dispatches via ``reorganize_album`` which handles file move + post-processing + DB update + sidecar through the same code path the per-album modal uses Job's per-album loop: - iterate albums for the active media server only (matches the artist- detail modal's scope; multi-server users won't have the job touch the inactive server's files at paths they can't see) - preview each album, catch exceptions per-album so one bad row doesn't abort the scan - branch on planner status: - ``no_album`` / ``no_tracks`` (race: album deleted mid-scan) → skip silently - ``no_source_id`` (album never enriched) → emit ONE album-level "needs enrichment first" finding (vs N per-track findings cluttering the UI) - ``planned`` → filter mismatched tracks (matched + new_path + not unchanged + file_exists), emit per-track findings (dry-run) or collect album for bulk enqueue (apply) - bulk enqueue at end of loop using the queue's correct return-shape (``{'enqueued': N, 'already_queued': M, 'total': K}``) What's gone (~500 LOC): - ``_read_tag_metadata`` / ``_get_audio_quality`` / transfer-folder walk - ``_load_album_years`` / ``_lookup_years_from_api`` (planner does this) - ``_apply_path_template`` / ``_build_path_from_template`` - direct ``shutil.move`` + sidecar move logic (queue handles) - the fragile ``is_album = group_size > 1`` heuristic — structurally gone - ``move_sidecars`` setting (no longer applicable; queue's post-process re-downloads cover art at the destination) What stays: - dry-run vs apply toggle - ``file_organization.enabled`` gate - stop / pause respect - progress reporting - findings for the UI Cleaner separation of concerns: - this job: DB-known tracks at wrong paths (active server only) - ``orphan_file_detector``: files on disk with no DB entry - ``dead_file_cleaner``: DB entries pointing to nonexistent files Tests: 12 tests in ``tests/test_library_reorganize.py`` pin the delegation contract — every status branch, every track-filter case, exception handling, apply-mode enqueue payload, active-server scope, estimate-scope shape. Three obsolete ``_lookup_years_*`` tests removed (year handling moved to planner). Closes #500 (the misclassification half — orphan + dead-file are downstream sync-gap symptoms, separate concern).	1 month ago
Broque Thomas	cceffbd8ec	Honor manually-matched source IDs in per-source enrichment workers ... GitHub issue #501 (@Tacobell444). After manually matching an album to a specific source ID via the match-chip UI, clicking "Enrich" on that album would fuzzy-search by name and overwrite the manual match with whatever the search returned — or revert the match status to ``not_found`` if name search missed. Reorganize then read the now- wrong ID and moved files to the wrong destination. Root cause was in the per-source enrichment workers' ``_process_*_individual`` methods. Several workers (Spotify, iTunes) ran search-by-name unconditionally with no check for an existing stored ID. Others (Deezer, Tidal, Qobuz) skipped on existing-ID but without refreshing metadata — preserved the ID but didn't actually honor the user's intent of "use this match to pull fresh data". Cin-shape lift: same fix needed in 5 workers, so extracted the shared behavior into ``core/enrichment/manual_match_honoring.py``: honor_stored_match( db, entity_table, entity_id, id_column, client_fetch_fn, on_match_fn, log_prefix, ) -> bool Per-worker variability (DB column name, client fetch method, response shape) plugs in via callbacks. Workers call the helper at the top of ``_process_album_individual`` / ``_process_track_individual``; if it returns True, the manual match was honored and the search-by-name fallback is skipped. If False (no stored ID, fetch failed, or empty response), the worker's existing search-by-name flow runs as before. Workers wired: - spotify_worker — album + track (was overwriting; now honors) - itunes_worker — album + track (was overwriting; now honors) - deezer_worker — album + track (was skip-on-id; now refreshes) - tidal_worker — album + track (was skip-on-id; now refreshes) - qobuz_worker — album + track (was skip-on-id; now refreshes) Workers left alone (already correct): - discogs_worker — already had inline stored-ID fast path that refreshes metadata. Same behavior, just inline; refactoring to use the shared helper would be churn for zero behavior change. - audiodb_worker — same — inline fast path with full metadata refresh. - musicbrainz_worker — preserves existing MBID and marks status, which is the correct behavior for MB (the MBID itself is the match payload — no separate metadata fetch). - lastfm_worker / genius_worker — name-based services with no source-specific IDs to honor. Inherent re-search per call. Reorganize fixed indirectly — it always honored stored IDs correctly via ``library_reorganize._extract_source_ids``. The "Reorganize broken" symptom was downstream of broken Enrich corrupting the stored ID. Tests: - ``tests/enrichment/test_manual_match_honoring.py`` — 11 tests pinning the shared helper contract: stored-ID fast path, no-ID fallthrough, empty-string treated as no ID, missing row, fetch exception caught and falls through, fetch returns None falls through, callback exceptions propagate, configurable table + column, defensive table-name whitelist. - Per-worker wiring NOT tested individually — the workers depend on live DB / client objects that are heavy to mock. The shared helper's contract is pinned; per-worker call sites are short enough to verify by code review. 2173/2173 full suite green. Closes #501.	1 month ago
Broque Thomas	fd5ccf4cb8	Fix "no such table: hifi_instances" via defensive lazy-create ... GitHub issue #503 (@hadshaw21). Adding a HiFi instance via downloader settings popped up ``no such table: hifi_instances`` even though "Test Connection" and "Check All Instances" both worked. Root cause: ``MusicDatabase._initialize_database`` runs every ``CREATE TABLE`` + every migration step inside one sqlite transaction. Python's sqlite3 module doesn't autocommit DDL by default, so if any later migration step throws on a user's specific DB shape (e.g. an old volume from a prior SoulSync version with quirky schema state), the WHOLE batch rolls back — including the ``hifi_instances`` CREATE that ran earlier in the function. The user's next boot retries init, hits the same migration failure, rolls back again. The ``hifi_instances`` table never lands no matter how many restarts. Fix: defensive lazy-create. New ``_ensure_hifi_instances_table(cursor)`` helper runs ``CREATE TABLE IF NOT EXISTS`` on demand, called immediately before every CRUD operation that touches ``hifi_instances``: - ``get_hifi_instances`` / ``get_all_hifi_instances`` (read) - ``add_hifi_instance`` / ``remove_hifi_instance`` (CRUD) - ``toggle_hifi_instance`` / ``reorder_hifi_instances`` (CRUD) - ``seed_hifi_instances`` (defaults seed) Idempotent — costs one no-op CREATE check when the table is already present, fully recovers from a broken init state. Read methods now return empty instead of raising when init failed; write methods work end-to-end. Doesn't paper over the underlying init issue (still worth tracking which migration step breaks for which user DB shapes — separate concern) but makes HiFi instance management self-healing in the meantime. Tests: - 7 obsolete tests that pinned ``raises sqlite3.OperationalError`` removed — that contract is no longer correct - 7 new tests pin the lazy-create behavior: every CRUD method works against a DB that's missing the ``hifi_instances`` table, verifying the table gets created and the operation completes 2162/2162 full suite green. Pure additive — no behavior change for users with a healthy DB; affected users get back to working hifi instance management. Closes #503.	1 month ago
Broque Thomas	9f2813fce4	Add cross-section dedup to all-libraries listing layer ... Followup to the all-libraries-mode commit. Without dedup, a Plex Home family where two users both have "Drake" in their music libraries would see "Drake" twice in SoulSync's library list — Plex returns distinct ratingKeys for each section's copy of the same artist. Dedup design — applied selectively, NOT everywhere: - ``_dedupe_artists(artists)``: groups by lowercased title, picks the canonical entry by ``leafCount`` (more tracks wins). Active ONLY in all-libraries mode; single-library mode is a no-op fast path with zero behavior change. - ``_dedupe_albums(albums)``: same but keys on (lowercased parentTitle, lowercased title) so two artists with identically-titled albums (e.g. self-titled releases) stay separate. Applied to: - ``get_all_artists()`` — public listing for the library view - ``get_library_stats()`` — count matches what user sees in the list Deliberately NOT applied to: - ``get_all_artist_ids()`` / ``get_all_album_ids()`` — these feed removal detection (compare returned ratingKey set against DB-linked ratingKeys to decide what's been removed). Deduping here would falsely flag non-canonical ratingKeys as "removed" and prune SoulSync's DB tracks that are linked to them. Pinned by two CRITICAL tests. - ``_all_tracks()`` — track count stays raw because the same track in two sections IS two distinct files / Plex entries, not a logical duplicate. - ``_search_general()`` and ``search_tracks`` Stage 1/2 — search results stay raw so cross-section matches aren't lost. Stage 1 may miss cross-section tracks for the same artist but Stage 2's server-wide track search catches them. Logging: when raw vs deduped artist counts differ, ``get_all_artists`` logs both so users can see "Found 4697 artists across all music sections (4521 unique after cross-section dedup)" — surfaces the overlap clearly. Tests: 8 new tests in test_plex_all_libraries.py pin: - canonical pick by leafCount (artists + albums) - case-insensitive name match - single-library no-op path (zero behavior change for those users) - album dedup keys on (artist, title) so different-artist same-title albums stay separate - ``get_all_artists`` listing applies dedup - ``get_all_artist_ids`` does NOT dedup (CRITICAL — removal detection) - ``get_all_album_ids`` does NOT dedup (CRITICAL — removal detection) - ``get_library_stats`` uses deduped counts for artists/albums but raw count for tracks Existing pre-stat test updated to use distinct mock instances — ``[MagicMock()] * 5`` creates five references to one mock which now correctly collapses under dedup. 71/71 media_server tests green, 2162/2162 full suite green. Honest known limitation acknowledged in WHATS_NEW + version modal: write-back (genre / poster / metadata updates) targets one ratingKey at a time — only updates the canonical section's copy of an artist if it exists in multiple. Other section's copy stays unchanged. Document and revisit if it matters.	1 month ago
Broque Thomas	620c41f1ac	Add "All Libraries (combined)" mode to PlexClient ... GitHub issue #505 (PopeBruhLXIX): users with multiple Plex music libraries (e.g. one per Plex Home user, or two folder roots split across separate library sections) only saw one library inside SoulSync because the connection settings forced you to pick a single library section. SoulSync's PlexClient stored exactly one ``self.music_library`` section reference and every read scanned only that one. This change adds an opt-in "All Libraries (combined)" dropdown option that flips the client into a server-wide read mode where every read method (``get_all_artists`` / ``get_all_album_ids`` / ``search_tracks`` / ``get_library_stats`` / etc) dispatches through ``server.library.search(libtype=...)`` instead of querying a single section. One Plex API call replaces N per-section iterations; Plex handles the aggregation server-side. Implementation: - ``ALL_LIBRARIES_SENTINEL`` (``'__all_libraries__'``) — module-level constant used as the saved DB preference value when the user picks the synthetic "All Libraries" entry. Detection is one string compare in ``_find_music_library`` / ``set_music_library_by_name``. Existing preferences (real library names) are unaffected. - ``self._all_libraries_mode`` (private flag) + ``is_all_libraries_mode()`` (public accessor for external callers). When True, ``music_library`` may stay None — ``is_fully_configured()`` recognizes the mode and still returns True so dispatch sites don't bail. - New private helpers ``_can_query``, ``_get_music_sections``, ``_all_artists``, ``_all_albums``, ``_all_tracks``, ``_search_general``, ``_search_artists_by_name``. Single dispatch point for the section-vs-server branch — every read method funnels through them so future drift fails at one place. - New public helpers for downstream callers: - ``get_recently_added_albums(maxresults, libtype)`` — used by DatabaseUpdateWorker's deep-scan recent-content sweep - ``get_recently_updated_albums(limit)`` — same - ``get_music_library_locations()`` — returns folder roots, used by web_server.py's file-path resolver - ``trigger_library_scan`` and ``is_library_scanning`` fan out across every music section in all-libraries mode. - ``get_available_music_libraries`` prepends a synthetic ``{'title': 'All Libraries (combined)', 'value': sentinel}`` entry ONLY when more than one music library exists. Single-library users don't get the extra option. ``value`` field is the canonical identifier the frontend submits to ``/api/plex/select-music-library`` (real libraries: title; synthetic: sentinel string). Backward- compatible — entries without ``value`` fall back to ``title``. Three crash points fixed in downstream consumers (would have failed during a deep scan after the user picked all-libraries mode): 1. ``database_update_worker.py:411`` — bailed out with "No music library found in Plex" because ``not self.media_client.music_library`` evaluated True in all-libraries mode (music_library is None there). Now uses ``is_fully_configured()`` which recognizes the mode. This was the root cause of the deep scan never starting. 2. ``database_update_worker.py:_get_recent_albums_plex`` — reached ``self.media_client.music_library.recentlyAdded()`` / ``.search()`` directly, AttributeError in all-libraries mode. Now routes through the new helper methods. 3. ``web_server.py:10947`` (file-path resolver) — accessed ``music_library.locations``; gated on ``music_library`` truthy so it didn't crash, but silently skipped all-libraries-mode locations. Now uses ``get_music_library_locations()`` which unions across sections. Plus polish: - ``/api/plex/clear-library`` also resets ``_all_libraries_mode`` so a fresh "select library" flow doesn't inherit stale mode state. - ``/api/plex/music-libraries`` surfaces "All Libraries (combined)" as ``current_library`` when in mode (settings UI displays correctly). - Frontend ``loadPlexMusicLibraries`` uses ``library.value || library.title`` so the sentinel-keyed option submits the sentinel string, not the human-readable label. Pre-select match handles both paths. Honest tradeoffs (documented as known limitations): - Same artist appearing in multiple Plex sections shows as separate entries in SoulSync (no dedup). Plex returns distinct ratingKeys for each. Cosmetic; revisit if it bites users. - Write-back (genre / poster updates) targets one ratingKey at a time — only updates that section's copy. Other sections' copies stay unchanged. - All-libraries mode includes any audiobook library that Plex classifies as ``type='artist'``. Edge case, opt-in only. Tests: 21 new tests in tests/media_server/test_plex_all_libraries.py pin both single-library mode (regression guard) and all-libraries mode for every refactored method. Existing test_plex_pinning.py fixture updated to initialize the new flag. 63/63 media_server tests green, 2148/2148 full suite green.	1 month ago
Broque Thomas	822759740d	Fix Download Discography pulling wrong artist + log routing ... Two fixes. (1) Discography endpoint now does server-side per-source ID resolution. When the user clicked Download Discography on a library artist, the endpoint received whichever artist ID the frontend happened to pick (spotify_artist_id || itunes_artist_id || deezer_id || library_db_id) and dispatched it as-is to whichever source it queried. If the picked ID didn't match the queried source's ID format, the lookup returned wrong-artist results (numeric ID collisions) or fell back to a fuzzy name search that picked a wrong artist. Two reproducible cases: - 50 Cent's library row had DB id 194687 — coincidentally a real Deezer artist ID for "Young Hot Rod". When the frontend's /enhanced fetch silently fell back to the DB id, the backend sent 194687 to Deezer, and Deezer returned Young Hot Rod's 50 albums in 50 Cent's discography modal. - Weird Al's library row had a stored Spotify ID. The frontend sent that to Deezer, which rejected the alphanumeric ID and fell back to fuzzy name search — which picked The Beatles somehow, returning 45 Beatles albums. The mechanism for per-source ID dispatch already exists in ``MetadataLookupOptions.artist_source_ids``, and the watchlist scanner already uses it; the on-demand discography endpoint just wasn't wired to it. Fix: when the URL artist_id matches a library row by ANY stored ID (DB id, spotify_artist_id, itunes_artist_id, deezer_id, or musicbrainz_id), pull every stored provider ID and pass them as ``artist_source_ids``. Each source gets its OWN stored ID regardless of which one the URL carries. When the URL ID is a non-library source-native ID and the row lookup misses entirely, behavior is identical to before (single-ID dispatch fallback). Logged the resolved per-source ID dict at INFO so future "wrong artist showed up" diagnostics are immediately legible in app.log. (2) Logger namespace fix in core/artists/quality.py and core/metadata/multi_source_search.py. Both modules used ``logging.getLogger(__name__)`` which resolves to ``core.artists.quality`` / ``core.metadata.multi_source_search`` — neither under the ``soulsync`` namespace where the file handler is wired. Result: every [Enhance], [MultiSourceSearch], and direct-lookup INFO line was being written to a logger with no handlers and silently dropped. App log showed the slow-request warning but no diagnostic detail. Switched both to ``get_logger()`` from utils.logging_config so the soulsync.* namespace picks them up. Same content, now actually lands in app.log. Confirmed working in live test: ``[Enhance] Direct lookup matched: deezer ID 1476162252 → 'Desastre'`` No behavior change in any other caller. Empty ``artist_source_ids`` (no library row matched) reaches lookup as ``None`` → identical to current single-ID dispatch path. Logger fix is pure routing — no content change.	1 month ago
Broque Thomas	3befe9349c	Direct ID lookup in Enhance Quality, like Download Discography ... Followup on the previous Enhance refactor. Multi-source parallel text search closed the worst case (users with no Spotify/Deezer getting "unknown artist - unknown album - unknown track" wishlist entries), but text search itself is still fragile against messy library tags: "Title (Live)", featured artists in the artist field, etc. Download Discography never had this problem because it resolves albums by stable ID, not by name. Enhance now does the same thing for tracks: for every metadata source the user has configured, if the library track has the corresponding stored ID (spotify_track_id / deezer_id / itunes_track_id / soul_id), call client.get_track_details(stored_id) directly and convert to the wishlist payload. First success wins. The user's configured primary source is tried first so a Deezer-primary user gets Deezer payloads on the wishlist entry (correct cover art / album shape) even when other sources also have stored IDs for the same track. Multi-source parallel text search stays as the fallback for tracks with no stored IDs (e.g. manually imported, never enriched). Empty- field rejection still gates the wishlist add. Implementation: - _STORED_ID_COLUMNS: source name → DB column mapping (Discogs intentionally omitted — release-based, no per-track IDs) - _enhanced_to_wishlist_payload: converts the get_track_details intermediate "enhanced" shape (artists as [str]) to wishlist shape (artists as [{'name': str}]). Spotify's raw_data is already in wishlist shape, returned as-is when detected (preserves full album.images that the enhanced top-level fields drop) - _try_direct_lookup_all_sources: iterates sources preferred-first, calls get_track_details on each that has both a stored ID and a configured client, returns first complete-metadata payload - spotify_client field removed from ArtistQualityDeps (no longer used — Spotify direct lookup now flows through the generic per-source loop using the entry from search_sources) - _try_upgrade_to_rich_payload removed (was Spotify-only with broken shape semantics for non-Spotify sources; search-fallback now uses _build_payload_from_track consistently) - get_primary_source() consulted to set the per-call preferred source for direct-lookup priority Also fixed a stale UI string: the Enhance modal toast read "Matching tracks to Spotify and adding to wishlist..." regardless of which sources were actually configured. Now reads "Matching tracks across metadata sources...". Tests: - _build_deps mirrors web_server._resolve_search_sources: passing spotify=spotify_obj auto-prepends ('spotify', spotify_obj) to search_sources (Spotify is always added when configured in prod) - 5 new tests pin the direct-lookup behavior: - test_direct_lookup_via_deezer_id_skips_text_search - test_direct_lookup_via_itunes_id_skips_text_search - test_direct_lookup_prefers_user_primary_source - test_direct_lookup_falls_through_to_text_search_when_no_stored_ids - test_direct_lookup_failure_falls_through_to_text_search - Reframed enhanced-format and search-fallback tests for the new payload-build path (no album-image side call, search-fallback uses _build_payload_from_track consistently) - 22/22 quality tests green, 2133/2133 full suite green.	1 month ago
Broque Thomas	7316646b01	Extract multi-source search; Enhance Quality matches Redownload coverage ... Track Redownload had been doing parallel multi-source metadata search across every configured source the whole time; Enhance Quality was running a single-source primary fallback that returned junk matches with empty fields when the primary was iTunes (Discord report: "unknown artist - unknown album - unknown track" wishlist entries for users with neither Spotify nor Deezer connected). Lift the redownload search into core/metadata/multi_source_search.py and point both flows at it. Same scoring, same per-source query optimization (Deezer's structured artist:/track: form), same current-match flagging via stored source IDs. ArtistQualityDeps now takes get_metadata_search_sources (returns [(name, client), ...] for every configured source) instead of the single-primary get_metadata_fallback_client + get_metadata_fallback_source. Spotify direct-lookup stays as a fast-path optimization (only Spotify exposes get_track_details(id) returning rich raw payload); when it doesn't fire, the multi-source parallel search picks the cross-source best match. Empty-field matches still rejected before wishlist add. Tests: _build_deps helper updated to accept the new search_sources contract while preserving fallback_client/fallback_source ergonomics. Reframed tests for the new semantics — direct-lookup is no longer gated on Spotify being the active primary; failure reason now lists every searched source. Added a test pinning the no-sources-configured prompt. 17/17 quality tests green, 2128/2128 full suite green.	1 month ago
Broque Thomas	4a27f3c245	Source-agnostic Enhance Quality flow + reject empty matches ... Discord report: clicking Enhance Quality on an artist with neither Spotify nor Deezer connected added tracks to the wishlist as "unknown artist - unknown album - unknown track". Root cause was structural. core/artists/quality.py had a hardcoded Spotify-direct → Spotify-search → iTunes-fallback chain that ignored the user's configured primary metadata source. When Spotify wasn't connected, every track fell through to an iTunes-only fallback that occasionally returned matches with empty fields (cleared the 0.7 confidence threshold but missing artist / album / title). Those empty strings propagated through the wishlist payload normalizer's truthy-check passthrough at core/wishlist/payloads.py:77-80 and the UI rendered them as "Unknown" defaults. Rewrote the flow source-agnostic: - ArtistQualityDeps gains get_metadata_fallback_source. Flow resolves the user's active primary source once up front. - New _build_payload_from_track helper produces the Spotify-shaped wishlist payload from any source's Track object — single place that knows how to construct it (replaces the duplicate construction in the Spotify-search and iTunes-fallback paths). - New _search_match helper does generic confidence-scored search against any client implementing search_tracks(query, limit). Same 0.7 threshold, same album-bonus weighting as before. - New _has_complete_metadata validator rejects matches with empty title / album / artists before they reach the wishlist. - _spotify_direct_lookup kept as a Spotify-only optimization (only Spotify exposes get_track_details(id) returning rich raw payload); other sources fall through to search. - Failure reason now names the active source: "No usable {source} match — connect another metadata source for better coverage". Result: Discogs users get a Discogs search. Hydrabase users get a Hydrabase search. iTunes users get an iTunes search with empty-field rejection. Spotify keeps its direct-lookup fast path. 6 new tests pin the architectural change: - Primary-source dispatch routes to the configured client (Discogs, not Spotify) when Spotify isn't primary - Spotify direct-lookup is gated on Spotify being the active primary (skipped when Discogs is configured even if track has spotify_track_id) - Empty title / album / artists fields all reject the match - Failure reason names the active source	1 month ago
Broque Thomas	b0dc139b72	Sync WHATS_NEW with current engine surface ... The "Media Server Engine Foundation" entry was written when the engine still had safe-default routing wrappers for optional methods. Those were dropped in the honesty pass. Entry now matches reality: - Lists the actual engine surface (6 methods: client / active_client / active_server / is_connected / configured_clients / reload_config) instead of claiming "uniform safe defaults for optional methods" - References KNOWN_PER_SERVER_METHODS as the data-only listing (replaces the old OPTIONAL_METHODS naming) - Cites real test counts (42 total) instead of the stale 35 - Drops the "33+ dispatch sites" overclaim (was already partial); the actual framing is "uniform-shape chains lifted, ~18 server-specific chains stay explicit per the lift-what's-truly-shared standard"	1 month ago
Broque Thomas	f230c93890	Merge remote-tracking branch 'origin/dev' into refactor/media-server-engine ... # Conflicts: # core/matching_engine.py # services/sync_service.py # web_server.py # webui/static/helper.js	1 month ago
Broque Thomas	edb6d1bc33	Drop dead per-server class imports + update WHATS_NEW ... - services/sync_service.py: dropped unused PlexClient / JellyfinClient / NavidromeClient class imports. After the engine refactor the service only needs TrackInfo for type annotations; the class imports were dead. - WHATS_NEW: extended the media server engine review-pass entry to cover the followup commits (Cin-5 per-server global removal + Gap 1 shared types lift) so the changelog matches the actual branch state.	1 month ago
Broque Thomas	d3f8a06d7a	WHATS_NEW entry for media server engine review pass	1 month ago
Broque Thomas	2c0a0da9ea	Address Copilot doc-drift review ... Four stale doc/comment references caught by Copilot's pass: - core/download_plugins/base.py: TYPE_CHECKING comment said the shared dataclasses lived in core.soulseek_client. They were moved to core.download_plugins.types in this PR. Comment updated. - core/qobuz_client.py: reload_credentials docstring still referenced soulseek_client.client('qobuz') after the global rename to download_orchestrator. Updated to download_orchestrator.client(...). - webui/static/helper.js: the older WHATS_NEW entries for the plugin contract + engine refactor still claimed backward-compat self.<source> attributes were preserved. Followup commits in the same PR removed them. Each entry now flags the followup explicitly and points at the "Drop Backward-Compat Per-Source Attrs" entry above it so the changelog is internally consistent. - docs/download-engine-refactor-plan.md: Compatibility commitments section listed orchestrator.<source> attribute preservation as a guarantee. Cin's review pass removed those attrs (and renamed the global handle from soulseek_client to download_orchestrator) — both are breaking changes for in-tree callers (which were migrated) and in-flight branches (which will need to update). Section rewritten to document the actual outcome.	1 month ago
Broque Thomas	2aff3dc210	Filter SoundCloud previews at every entry point + fix hybrid fallback regression ... The earlier validation-only filter only ran in the auto-search scoring path. SoundCloud preview snippets still leaked through: - The candidate-review modal cached raw search results (pre-validation), so previews were visible and clickable for manual retry — and the manual-pick download path bypassed validation entirely, downloading the preview anyway. - The not-found raw-results cache stored unfiltered top-20s. Lift the preview filter into a reusable filter_soundcloud_previews() helper and apply it at every entry point: validation scoring (still), modal-cache fallback when validation drops everything, and the not-found raw-results path. Previews now never reach the cache, the matcher, or the manual-pick UI. Drops candidates < 35s or below half the expected duration, gated on expected > 60s so genuine short tracks still pass. 7 new unit tests pin the helper. Also fixed a silent regression in core/downloads/task_worker.py's hybrid-fallback path. Cin-5 dropped the per-source attrs from the orchestrator (orch.soulseek, orch.youtube, etc.), but the fallback loop still resolved sources via getattr(orch, '<src>', None) — every lookup silently returned None, so remaining_sources came back empty and the fallback never ran. Now uses orch.client(name) like the rest of the codebase. Updated the test fake to expose client() too — the old test was passing because the loop was effectively dead.	1 month ago
Broque Thomas	563204ceae	Drop SoundCloud preview snippets before scoring ... SoundCloud serves a ~30s preview clip for tracks gated behind Go+ or login (extremely common for major-label uploads — what's actually on SoundCloud is bootlegs, fan reuploads, type beats, and these previews). yt-dlp accepts the preview as the download payload, the post-download integrity check catches the duration mismatch and quarantines the file, but the user only sees "all candidates failed" with no obvious explanation. Filter at validation time when we know expected_duration: drop SoundCloud candidates whose duration is below half the expected length OR within ~5s of the 30s preview boundary, gated on expected being non-trivially long (>60s) so genuinely short tracks still pass through.	1 month ago