704 Commits (9656dbd46a4fb2fee9e28223318f5e0b0ed3c00a)

Author	SHA1	Message	Date
Antti Kettunen	9656dbd46a	Thread runtime through metadata enrichment ... - Pass the live runtime bundle into the shared metadata facade so worker-backed source enrichment can actually run. - Forward runtime from the import pipeline and web-server wrapper into embed_source_ids. - Add a regression test that verifies the runtime object reaches the source-ID embedding path.	4 weeks ago
Antti Kettunen	8319c6679f	Move new metadata helpers into a package ... - Keep existing metadata_cache and metadata_service at the top level for now - Move the new branch-local metadata helpers under core/metadata - Share MusicBrainz release cache state from core.metadata.source and update import sites	4 weeks ago
Antti Kettunen	bdef127dd6	Lift shared runtime state into core ... - Move app-wide task and activity registries out of core/imports - Share one runtime-state module across the web server, API, and import pipeline - Keep import-specific helpers focused on context and post-processing	4 weeks ago
Antti Kettunen	e10df4caf2	Rehome import helpers into core/imports ... - Move import flow modules into a dedicated package - Update app and test imports to the new namespace - Group the import-focused tests under tests/imports	4 weeks ago
Antti Kettunen	b9269b4f16	Tighten metadata helper boundaries ... - remove stale wrapper helpers from web_server and metadata_common - import provider helpers directly in metadata_source - keep the metadata modules' public surface explicit	4 weeks ago
Antti Kettunen	edd9048f86	Checkpoint metadata runtime cleanup ... - remove runtime from metadata helper APIs where it only carried config, logger, mutagen, and database access - keep runtime only for the source-ID enrichment path that still needs live worker handles - add the new metadata helper modules and update the tests to match the slimmer interfaces	4 weeks ago
Antti Kettunen	6872e5080d	Refine import module boundaries ... - Move filename and staging helpers into their canonical modules - Extract album naming and grouping from path handling - Update import and test call sites to the new layout	4 weeks ago
Antti Kettunen	0bbf44809f	Move the import flows and related post-processing pipelines into separate modules ... - Extract the import pipeline, album import, staging, path, file ops, guards, runtime state, side effects, and metadata enrichment out of . - Canonicalize the refactored import path around and remove legacy , , , and request shapes from the import endpoints. - Make album and track metadata lookups follow the configured provider priority instead of hard-coding Spotify, while still falling back when needed. - Update the import routes and frontend payloads to use the new core helpers. - Add coverage for the extracted helpers and the refactored import flows. PS. apologies to anyone who might check this commit out - the intention was to start small, but things kinda snowballed out of control at some point since the logic just kept going on and on, and everything kinda had to be changed all at once for it all to make any sense	4 weeks ago
Broque Thomas	dd4cf130d7	Socket.IO CORS: handle self-review nits ... Six items from a Cin-style line-by-line pass on PR #383: - resolve_cors_origins: list of non-string entries (`[None, 123]`) now drops them instead of coercing to junk strings like `'None'`/`'123'`. - will_reject: backwards-compat shim removed. Production callers always pass `request.scheme` (Flask-guaranteed); the shim only existed for tests/non-Flask callers and made the production code path branchier than necessary. Tests now pass scheme explicitly. - maybe_log: redundant `if not origin` early-return dropped. will_reject handles missing origin (engineio's own behavior — server.py:207). - RejectionLogger.__init__: `int(dedup_cap)` wrapped in try/except so bad-type input falls back to DEFAULT_DEDUP_CAP instead of raising. - web_server.py: docstring on the before_request hook explains why the hook fires on every request (Flask doesn't scope before_request to a path prefix; the early-return string compare is the cheapest option). - settings.js: cors-origins URL regex tightened from `[^\s/]+` to `[^\s/?#]+` so query/fragment chars don't pass validation. Engineio would silently fail to match those anyway; better to flag at save. Test changes: - parametrize gained an explicit `scheme` column (12 cases updated). - New explicit case: scheme-mismatch rejects (engineio compares full `{scheme}://{host}` strings). - `test_will_reject_falls_back_to_host_only_when_no_scheme_info` deleted — the shim it tested is gone. - `test_will_reject_honors_x_forwarded_host` now passes scheme info. Net: -9 production lines, -3 test lines. Production code path is straight-line. 603 tests pass.	4 weeks ago
Broque Thomas	0f24739e27	Socket.IO CORS: polish — match engineio exactly, bound dedup, validate URLs ... Self-review pass on the security fix uncovered five issues, all fixed here: 1. will_reject scheme handling. Engineio compares full {scheme}://{host} strings, not just hostnames. A TLS-terminating proxy can leave the backend seeing http while the browser's Origin is https — engineio rejects, but the original predictor said "allow" → no helpful log line. Added request_scheme + forwarded_proto params, build full candidate strings to match engineio. 2. EITHER-forwarded-header rule. Engineio adds the forwarded candidate when EITHER X-Forwarded-Proto OR X-Forwarded-Host is present (it falls back to HTTP_HOST for the missing one). The original predictor only added it when forwarded_host was set — false negative for misconfigs sending only X-Forwarded-Proto. Now mirrors engineio. 3. will_reject incorrectly rejected missing-Origin requests. Engineio (server.py:207: `if origin: validate`) skips CORS validation when no Origin header is sent — non-browser clients (curl etc.) are intentionally permitted. The original code rejected them. Test was asserting the wrong behavior. Both fixed. 4. RejectionLogger had unbounded dedup set growth. A hostile actor opening connections from many distinct fake origins would fill memory unboundedly. Capped at 100 unique origins (configurable); when cap hit, one overflow notice is emitted and further rejections are silently dropped until restart. 5. Lock pattern: the overflow log path called logger.warning() while holding the dedup lock, inconsistent with the normal path. Fixed to pick the message under the lock and log after release. Critical section is now minimal and uniform. Plus polish: - Stale module docstring fixed (said "empty list" instead of "None"). - settings.js validates each cors_origins line against a URL regex on save; toasts a one-shot warning if entries are malformed (resolver silently filters them, but user gets feedback now). - web_server.py wiring passes request.scheme + X-Forwarded-Proto so the predictor has full proxy info. Tests: - 51 unit tests in tests/test_socketio_cors.py (was 45). New cases: * scheme comparison (5 cases including TLS-terminating proxies) * forwarded_proto-alone misconfig * missing-origin matches engineio (was asserting wrong behavior) * dedup cap with overflow + reset * default cap is reasonable (uses public DEFAULT_DEDUP_CAP constant) Engineio behavior independently verified by reading engineio/server.py and engineio/base_server.py source. Predictor mirrors both files. 604 tests pass.	4 weeks ago
Broque Thomas	013eebf350	Lock down Socket.IO CORS — same-origin default + opt-in allow-list ... Closes #366 (reported by JohnBaumb). Socket.IO was initialized with `cors_allowed_origins='*'`, accepting WebSocket connections from any origin. A malicious site could open a WS to a user's local SoulSync instance and exfiltrate live progress / toast / activity events. This commit: - Defaults to engineio's same-origin behavior (`cors_allowed_origins=None`), which automatically honors X-Forwarded-Host so reverse proxies that send that header (Caddy / Traefik by default, properly-configured Nginx) work transparently. - Adds a `security.cors_origins` config setting + Settings → Security textarea where users behind unusual proxies / Electron wrappers / cross-origin integrations can whitelist their origin. Accepts comma or newline separated values; `*` on its own line opts back into the legacy wildcard with a startup-warning log. - Logs a clear warning the first time engineio rejects each unique origin, naming the rejected Origin and request Host and pointing users to the settings field. Without this, engineio silently 403s the upgrade and the user just sees a half-broken UI with no clue why. Threadsafe dedup so a hostile origin can't spam logs. Logic lives in `core/socketio_cors.py` (resolver, rejection predictor, dedup logger class, startup-status emitter) — pure functions, no Flask dependency. `web_server.py` adds 23 lines of wiring and imports. Important catch during review: my first pass used `cors_allowed_origins=[]` as the "secure default." Reading engineio's source revealed `[]` actually means "DISABLE CORS HANDLING" (engineio/server.py:202: `if cors_allowed_origins != []:`) — identical security to `'*'`. Fixed to use `None` (engineio's actual same-origin sentinel) and pinned with a regression test that asserts the resolver never returns `[]` for any input shape. Tests: - tests/test_socketio_cors.py — 45 unit tests covering 19 resolver shape cases (None, empty, whitespace, comma, newline, garbage types, lists), the `[]`-must-never-be-returned security regression, 12 rejection prediction cases, X-Forwarded-Host handling, dedup logger behavior, threadsafe race (8 threads × 50 hammers → exactly 1 warning), and startup-status emitter outputs. Frontend: - Settings → Security gains an "Allowed WebSocket Origins" textarea with help text explaining same-origin default + when to add a domain + the `*` opt-out. - helper.js — new '2.4.1' WHATS_NEW block (hidden until version bump) with a chill-voice entry describing the change. Conftest.py left at `'*'` — test environment, no security concern. 598 tests pass.	4 weeks ago
Broque Thomas	37aefd2ff1	Reorganize queue: race + dedupe fixes from kettui review ... Five issues kettui flagged on PR #377: - Worker race (reorganize_queue.py): _next_queued() picked an item and released the lock, then re-acquired to flip status='running'. A cancel() landing in that window marked the item cancelled but the worker still ran it. Replaced with _claim_next_or_wait() that picks AND flips under one lock acquisition. - Wakeup race (reorganize_queue.py): _wakeup.clear() after the empty check could lose an enqueue's _wakeup.set(), parking a freshly-queued album for up to 60 seconds. Replaced Lock + Event with a single threading.Condition; cond.wait() releases and re-acquires atomically on notify. - Bulk dedupe (reorganize_queue.py:enqueue_many): looped single-item enqueue, so a duplicate album_id later in the same batch could slip through if the worker finished the first copy before the loop reached the second. Now holds the lock for the whole batch and tracks a per-batch seen set, so intra-batch duplicates dedupe against each other and not just pre-existing items. - Preview button stuck disabled (library.js:loadReorganizePreview): early returns and thrown errors skipped the re-enable line. Moved state into a canApply flag committed in finally, so any exit path lands the button correctly. - DB helpers swallowing failures (music_database.py): get_album_display_meta and get_artist_albums_for_reorganize used to catch every Exception and return None / [], so a real DB outage masqueraded as "album not found" / "no albums". Now lets exceptions bubble; the route layer already wraps them as 500. Tests: - test_cancel_and_run_are_mutually_exclusive — hammers enqueue+cancel pairs and asserts the invariant that no successfully-cancelled item ever ran (catches regressions to the atomic pick). - test_enqueue_many_dedupes_batch_internal_duplicates — pins the intra-batch dedupe. - test_get_album_display_meta_propagates_db_errors and test_get_artist_albums_for_reorganize_propagates_db_errors — pin the bubble-up behavior. Changelog updated in helper.js and version modal.	4 weeks ago
Broque Thomas	d6094a3587	Library reorganize: FIFO queue with live status panel ... Replaces the single-slot "one reorganize at a time, return 409 on collision" model with a per-user FIFO queue. Buttons stay clickable, "Reorganize All" is one backend call instead of an N-call JS loop, and a status panel mounted at the top of the artist actions bar shows live progress (active item, queued count, recent completions) with per-item cancel buttons. Backend - core/reorganize_queue.py: singleton queue + worker thread, dedupe-on- enqueue, cancel rules (queued cancellable, running not), enqueue_many for bulk operations, progress fan-out via update_active_progress - core/reorganize_runner.py: factory builds the worker's runner closure with injected dependencies. Reads config per-call so changing the download path in Settings takes effect on the next reorganize without a server restart - database/music_database.py: get_album_display_meta and get_artist_albums_for_reorganize — moves the SQL out of route handlers - web_server.py: thin enqueue/snapshot/cancel/clear endpoints, runner registration at module load. Old _reorganize_state globals + status endpoint deleted. Static-asset cache buster (?v=<server-start>) added so JS/CSS updates ship live without users clearing cache Frontend - webui/static/library.js: status panel mount, polling (1.5s when active, 8s when idle), expand/collapse, per-item cancel, debounced enhanced-view reload (one reload per artist batch instead of N). Per-album reorganize button paints with queued/running indicator and short-circuits to a toast when the album is already in queue - webui/static/style.css: panel + button styling matching the existing glass-UI accents - webui/static/helper.js + version modal: WHATS_NEW entry Tests (22 new) - tests/test_reorganize_queue.py (19 tests): FIFO order, dedupe, per-item source, cancel rules, continue-on-failure, snapshot shape, progress propagation, bulk enqueue - tests/test_reorganize_runner.py (4 tests): per-call config reads, setup-failure summary, dependency injection, progress fan-out - tests/test_reorganize_db_methods.py (7 tests): SQL JOIN behavior, ordering, fallback for blank strings, artist isolation Full suite 549 passed in 27s.	1 month ago
Broque Thomas	98c85f928e	Merge remote-tracking branch 'origin/dev' into fix/reorganize-via-post-process-pipeline ... # Conflicts: # webui/static/helper.js	1 month ago
Broque Thomas	7e1c4c26ec	Reorganize: fix moved-count + status/total UX issues from PR #377 review ... Four changes addressing kettui's PR #377 review comments: 1. **`_finalize_track` no longer over-counts on DB failure (🔴 bug).** The function previously bailed on DB-update failure but `_process_one_track` still incremented `summary['moved']` unconditionally — overstating how many tracks the UI knows are at their new locations. Fixed by: - `_finalize_track` now returns ``bool`` (True only when DB row was updated AND original was dealt with) - Caller checks the return; on False, records as a failed track with a clear message ("Track landed at new location but DB update failed — file is at both old and new paths until library scan re-indexes") - Existing `test_db_update_failure_leaves_original_in_place` now also asserts `moved == 0`, `failed == 1`, and that the error message names the cause 2. **`executeReorganize` toast no longer says "undefined tracks" (🐛 bug).** `/reorganize` doesn't return `result.total` anymore (the track count is determined server-side after planning), so the "Reorganizing undefined tracks..." string was meaningless. Now uses `result.message` from the backend instead. 3. **`_pollReorganizeStatus` distinguishes completed from skipped (🟡 risk).** Backend now propagates the orchestrator's status (`completed` / `no_source_id` / `no_album` / `no_tracks` / `setup_failed` / `error`) into `_reorganize_state['result_status']` so the frontend can warn appropriately. Two new helpers: - `_classifyReorganizeOutcome(state)` — returns 'success' only when `result_status === 'completed'` AND `failed === 0`; 'warning' otherwise - `_formatReorganizeResultMessage(state)` — returns a message specific to the outcome ("Reorganize skipped — album has no metadata source ID. Run enrichment first." for `no_source_id`, etc.) Zero-failure non-completed runs now show as warnings instead of green checkmarks. 4. **Bulk mode no longer counts skipped albums as succeeded (🟡 risk).** `_executeReorganizeAll`'s loop was treating any HTTP 200 response as success, ignoring the orchestrator's actual outcome for that album. Fixed by: - `_waitForReorganizeComplete()` now resolves with the final state object (was: void) - Loop checks `finalState.result_status === 'completed'` AND `finalState.failed === 0` before counting `succeeded++`; otherwise increments `skipped` (with a per-album warning toast) or `failed` accordingly - Final summary toast now reads "Reorganized N of M albums, K skipped, J failed" and only shows green when nothing was skipped or failed All four addressed in a single commit because they form one coherent UX-correctness fix — the bug bug (#1) and the count- overstatement bug (#4) both made the user see "everything succeeded" when reality was different. Together they make the UI honestly reflect what actually happened. Files: - core/library_reorganize.py — `_finalize_track` returns bool, `_process_one_track` reads it - web_server.py — `_reorganize_state['result_status']` populated from orchestrator's summary on success and on exception - webui/static/library.js — `_classifyReorganizeOutcome` / `_formatReorganizeResultMessage` helpers, single-album + bulk-mode flows both consume them - tests/test_library_reorganize_orchestrator.py — strengthened the existing DB-failure test to assert moved/failed counts Credit: kettui — four PR #377 review comments named all of these precisely with line numbers and severity.	1 month ago
Broque Thomas	6c90d68de3	Discogs: count rows with empty type_ as real tracks too ... Reported by kettui on PR #374 review: the inline filter that backed `set_album_api_track_count` only counted rows where `type_ == 'track'`, but `discogs_client.get_album_tracks` itself accepts both `'track'` AND empty `type_` as real songs (line 660: `type_ in ('track', '')`). Releases where Discogs returns some real tracks with an empty `type_` field would be undercounted, which would silently disagree with the repair job's fallback `_get_expected_total` path (which calls into `get_album_tracks_for_source` and therefore uses the client's count). Extracted the filter into `count_discogs_real_tracks(tracklist)` — single source of truth for the rule, testable in isolation, and the worker call site is now a one-liner that names what it's doing. Also defensive about the input shape: `type_ == None`, missing field, and empty/None tracklist all handled cleanly. 10 tests pin the behavior: - empty/missing/None type_ all count as a real track (the kettui case) - 'heading', 'index', 'sub_track' excluded - unknown future type strings excluded conservatively - realistic multi-disc tracklist with mixed shapes counts correctly - empty/None input returns 0 without raising Credit: kettui — the PR #374 review comment that flagged this.	1 month ago
Broque Thomas	cb67773998	Merge remote-tracking branch 'origin/dev' into fix/album-completeness-api-track-count ... # Conflicts: # webui/static/helper.js	1 month ago
Broque Thomas	2b15260b88	Reorganize: route library files through the post-processing pipeline ... Reported on Discord by winecountrygames. The library "Reorganize" tool had several layered bugs that all traced to the same root cause: the endpoint reinvented every wheel post-processing already turns — its own template engine, its own disc-number resolution from file tags, its own sidecar sweep, its own collision detection — and each had drifted from the canonical path used by fresh downloads. Reported symptoms: - 3-disc Aerosmith deluxe collapsed to a flat single-disc layout - Half the tracks on other albums silently skipped, no error / no count - Re-runs left empty leftover album folders cluttering the artist dir Architecture: stop reinventing wheels. Route reorganize through exactly the same pipeline downloads use. Per-album: 1. Fetch the canonical tracklist from a metadata source (Spotify / iTunes / Deezer / Discogs / Hydrabase) using the album's stored source IDs. New `core/library_reorganize.py::plan_album_reorganize` does this — primary-source-first, fall through priority chain unless the user picked a specific source in the modal (strict mode). 2. For each local track, find the matching API entry via a scored candidate matcher. Score components: exact-title (100), substring-with-length-ratio (40-90), track-number agreement (20). Hard reject when the two titles have different version differentiators (Remix vs no-remix means different recordings, not annotation drift). Below threshold = unmatched, surfaced as "not in source's tracklist, left in place" rather than silently mis-routing. 3. Copy the file to a per-album staging directory, build the same context dict the import flow builds (`spotify_album` / `track_info` / etc. with `is_album_download=True` so the path builder enters ALBUM mode, not SINGLE mode), call `_post_process_matched_download(...)` — same function fresh downloads use. Post-process handles tagging, multi-disc subfolder decisions, sidecar regeneration, AcoustID verification. 4. Read `context['_final_processed_path']` to learn where it landed. Update `tracks.file_path` in the DB BEFORE removing the original (DB-update failure leaves the file at both locations, recoverable via library scan; the reverse would orphan the row). Delete per-track sidecars (post-process recreates them at the new destination). 3 concurrent workers per album via ThreadPoolExecutor, matching the download path's per-batch worker count. State mutations all guarded by a single lock; staging filenames carry a UUID prefix so concurrent copies of identically-named source files don't overwrite each other. Source picker in the modal lets the user choose which source to read the tracklist from. Two endpoints feed it: - `/api/library/album/<id>/reorganize/sources` — sources for THIS album that are both authed AND have a stored ID. For the per- album modal. - `/api/library/reorganize/sources` — all authed sources globally. For the bulk "Reorganize All" modal where per-album ID coverage varies. When the user picks a specific source, the orchestrator runs in `strict_source=True` mode (no fallback chain) — picking Spotify means "use Spotify or fail", not "use Spotify and silently fall back." Preview endpoint shares the same planning logic as apply via `preview_album_reorganize` — the destination path comes from the same `_build_final_path_for_track` post-process uses, so what you see in the preview is exactly what you get on apply. Empty destination folders (from earlier failed runs OR from the current run when post-process creates a dir then fails AcoustID) get cleaned up after each successful run: walk up to the artist folder from any successful destination, prune empty album-sibling folders one level deep. Bounded scope = won't touch unrelated user dirs. Web_server.py shrinks by ~450 net lines. The endpoint handler is now a thin wrapper that builds injected callables (path resolver, post- process function, DB updater, empty-dir cleaner), spawns a thread that calls `reorganize_album()`, and returns. All actual logic lives in `core/library_reorganize.py` where it's unit-testable without spinning up Flask. Frontend cleanup: the per-call template input in both reorganize modals (per-album and bulk) was redundant — the backend always uses the configured global download template. Removed the input and the variables-grid reference UI it was for. 39 new unit tests pin every contract: - source resolution (no_source_id when album has none, fallthrough chain when primary returns nothing, strict mode bypasses fallback) - matcher scoring (exact / substring / multi-disc disambiguation / smart-quote tolerance / dash-vs-parens / bonus-track substring / Remix-vs-original differentiator rejection / "Real" doesn't false- match "Real Real Real" / track-number-only no longer fires) - file safety (DB-update failure leaves original in place, post- process failure leaves original in place, post-process exception caught and original preserved, success removes original AND updates DB in the right order) - sidecar handling (per-track .lrc/.nfo deleted on success, kept on failure; album-level cover.jpg/folder.jpg cleaned only when directory has no remaining audio) - staging cleanup (recreated between tracks because post-process nukes it, dir cleaned up on success AND on failure) - destination-dir prune (empty siblings removed, real album with files preserved, no recursive sweep) - source picker (only authed-with-stored-ID sources for per-album, all authed sources for bulk; strict mode doesn't fall back) - concurrency (3 workers in flight, state stays consistent under races, stop_check cuts off pending tasks) - preview parity (preview produces same destination as apply for multi-disc; ALBUM mode not SINGLE mode; unmatched/no-path tracks surfaced with reasons) Limitations (deliberate punts, NOT in this PR): - Renamed local titles on multi-disc albums where track_number also disagrees: matcher returns nothing (track is "not in source"). Fixable by using duration_ms as a tertiary signal. - Per-track in-modal source switching with per-album track-count hints (would need a second API call before opening the modal). - UI status panel on the artist page during a run — currently just toasts. Documented as a follow-up PR. Files: - core/library_reorganize.py — new module: plan_album_reorganize, preview_album_reorganize, reorganize_album, available_sources_for_album, authed_sources, _score_candidate, helpers for staging/post- processing/finalizing, sidecar + dest-dir cleanup - core/metadata_service.py — no changes; reused get_album_for_source, get_album_tracks_for_source, get_source_priority, get_client_for_source - web_server.py — three endpoints (preview / apply / sources GETs) are thin wrappers; -450 net lines - tests/test_library_reorganize_orchestrator.py — 39 tests covering every contract above - webui/static/library.js — source picker UI in both modals; dead template input + variables-grid removed - webui/static/style.css — dropdown option styling fix (white-on- white was unreadable) Reported on Discord by winecountrygames — his bug report named the trigger button (Enhanced view → Reorganize All) and both symptoms (multi-disc collapse, half-album skip), which let the diagnosis go straight to the architectural problem.	1 month ago
Broque Thomas	252121ca96	Bump Spotify post-ban cooldown from 5 min to 30 min ... Reported on Discord by winecountrygames — Spotify auth granted, then re-banned for 4 hours within ~30 seconds, repeatedly. Trace from his captured log: < 12:05 [pre-log] Spotify ban active when log starts 15:21:27 First ban EXPIRED → 5-minute post-ban cooldown begins 15:26:27 Cooldown ends, spotify_client.is_authenticated() probe allowed again → client initialized 15:26:59 First Spotify API call after cooldown — get_artist_albums for an artist whose discography a background worker was enriching — gets 429 immediately with no Retry-After header → new ban activated for 14400s (4 hours) Root cause: `_POST_BAN_COOLDOWN = 300` (5 minutes) is shorter than Spotify's actual server-side memory of the previous offense. The cooldown exists specifically to prevent the "ban expires → we probe → re-ban" cycle (`spotify_client.py:65-68` documents that intent explicitly), but the value was wrong: Spotify's server still considered this user banned 5 minutes after our local ban window ended, so the very first call after cooldown got slapped. The 4-hour re-ban itself is correct behavior — `_BASE_MAX_RETRIES_BAN` fires when spotipy reports "max retries", which means the client exhausted its internal retry budget on 429s before raising. That's a severe-ban signal and a long default is the right response. Fix: bump `_POST_BAN_COOLDOWN` to 1800 seconds (30 min). This is the smallest change that addresses the immediate "re-probe → re-ban" loop in the report. 30 minutes is an empirical floor — long enough for Spotify to actually clear its server-side memory in the cases we've observed, short enough not to keep functional users locked out beyond necessary. Can be revisited if reports persist. What this PR does NOT fix (important context for the same user): This bump only helps the "ban expires → we re-probe → re-ban" loop. It does NOT help winecountrygames's other symptom — Spotify being banned within 30 seconds of his FIRST EVER authorization (no prior ban). That's a separate failure mode: on first auth, enrichment workers immediately fan out across the user's library (250 artists in his case), hammering Spotify endpoints with bulk get_artist_albums calls before any rate-limit feedback can land. Spotify's hidden per-endpoint daily quotas — which BoulderBadgeDad has empirically documented but the global rate limiter doesn't see — flag the burst and impose a multi-hour cooldown that LOOKS like a bot-detection ban to us. A proper fix needs a fresh-auth ramp-up: start with very low Spotify QPS for the first N minutes, scale up only if no rate-limit feedback arrives. That's a separate PR. Documented as additional follow-ups (NOT in this change): - Adaptive cooldown that scales with the size of the previous ban — a 4-hour MAX_RETRIES ban probably warrants a 1-hour cooldown, while a 60-second Retry-After-honored ban can resume in 5 minutes. The system already distinguishes these in `_set_global_rate_limit`, it just doesn't propagate the distinction to cooldown duration. - Probe-with-light-call pattern — make the first post-cooldown call a single inexpensive endpoint (`current_user`) rather than allowing a background worker's heavy `get_artist_albums` to be the canary. Failed probe extends cooldown silently instead of triggering a fresh 4-hour ban. - Fresh-auth ramp-up (per the limitation above). Files: - core/spotify_client.py — `_POST_BAN_COOLDOWN` 300 → 1800. Comment expanded to cite the report so the value isn't bumped back without context. - webui/static/helper.js — WHATS_NEW entry under 2.40 explaining the change for affected users. No tests added — the cooldown logic itself is unchanged, only the constant. Tests asserting on a constant value are theater. Reported on Discord by winecountrygames — his captured log made the "ban-expires-to-re-ban" timing chain unambiguous.	1 month ago
Broque Thomas	b3afed1599	Fix Tidal device-auth link opening SoulSync instead of link.tidal.com ... The "Link Tidal Account" device-flow UI displayed a verification URL like `link.tidal.com/XBXYT` that, when clicked, navigated back to the SoulSync origin (e.g. `http://localhost:8889/link.tidal.com/XBXYT`) instead of to Tidal's activation page. Root cause: tidalapi returns `login.verification_uri_complete` as a schemeless string. settings.js drops it straight into `<a href>`, and browsers treat schemeless hrefs as same-origin relative URLs. Normalize the URI in `start_device_auth` — if it doesn't already start with `http://` or `https://`, prepend `https://`. Same treatment for the `link.tidal.com/{user_code}` fallback so the defensive path stays well-formed too.	1 month ago
Broque Thomas	a9f827ef42	Reject Tidal streams that silently downgrade from the requested quality ... Reported on Discord by Netti93: with Tidal configured for "HiRes only" and "Allow Quality Fallback" disabled, tracks were still downloading successfully — as m4a 320kbps files. Some "successful" downloads were less than half the file size of the same track pulled via Tidarr/tiddl from the same Tidal account. Root cause: Tidal's API silently degrades to the best quality your account + the track + your region permits. Setting `session.audio_quality = Quality.hi_res_lossless` and calling `track.get_stream()` on a track that's only available in AAC returns an AAC stream with no error. The downloader wrote the m4a file to disk, the ~7MB size sailed past the 100KB stub threshold, and the download reported success. The pre-existing "verify quality wasn't silently downgraded" block only LOGGED a warning when this happened; it did not fail the tier. Two knock-on effects: - Users with "HiRes only, no fallback" got m4a files anyway, which defeats the setting entirely. - The worker-level fallback chain (hires → lossless → high → low) couldn't advance past the first tier, because every tier "succeeded" at whatever Tidal happened to serve. Fix: after `track.get_stream()`, compare `stream.audio_quality` against the tier we asked for using a rank-based ordering: LOW < HIGH < LOSSLESS < HI_RES < HI_RES_LOSSLESS - Same tier or higher → accept (so the occasional Tidal upgrade doesn't get rejected just because it's not an exact match). - Lower tier → reject THIS tier. The loop `continue`s and the next fallback tier is tried, or the whole download fails honestly when the user has fallback disabled. The existing final-error log already has a hint directing users to enable fallback if they want automatic Lossless substitution. - Unrecognized `audioQuality` value (e.g. a new Tidal tier we haven't mapped) → reject conservatively, so the next fallback tier gets a chance and the diagnostic log names the unknown value. Why the rank-based approach instead of strict equality: Tidal's API doesn't technically promise an exact-tier match on serving; on tracks that are flagged in its catalog as a higher tier, it can serve higher than the session setting. Rejecting higher-than-asked quality would be user-hostile. And the `HI_RES` (legacy MQA) value — not in tidalapi's modern `Quality` enum but possibly still present on old catalog entries — needs to rank below `HI_RES_LOSSLESS`: users asking for true lossless HiRes should reject MQA since MQA is a lossy format. tidalapi's `Quality` enum is a `str` subclass whose VALUES (not member names) match what the Tidal API returns in the `audioQuality` field (e.g. `Quality.hi_res_lossless.value == 'HI_RES_LOSSLESS'`, `Quality.low_320k.value == 'HIGH'`). Both sides of the comparison are coerced to `str` before use, so the check is robust to whichever tidalapi version exposes the served quality as an enum or a plain string. The check is extracted as `_verify_stream_tier(stream, q_info, q_key) -> (ok, reason)` at module scope — a pure function with no I/O, unit-tested independently. Ten tests: match, three upgrade cases (LOSSLESS → HI_RES_LOSSLESS, LOSSLESS → HI_RES, LOW → any higher), three downgrade cases (the reported HiRes → AAC, HiRes Lossless → MQA HiRes, Lossless → AAC), one unrecognized-tier case, and two defensive paths for older tidalapi builds without `audio_quality` on the stream object and for QUALITY_MAP entries that lack `tidal_quality` (e.g. tidalapi wasn't importable at module load). Test stub updated to use uppercase `Quality` values matching real tidalapi so case-sensitivity regressions get caught. Also removed the old codec-string-based warning block — the new tier check is strictly stronger, and keeping the warning around would just be dead code waiting to drift out of sync. Deliberately NOT tackling in this PR (documented as follow-ups): - Bit-depth verification of HiRes FLAC files via mutagen. The `stream.audio_quality` tier check catches the main "HiRes requested, got AAC" case; bit-depth would only matter if Tidal labeled a stream HI_RES_LOSSLESS but served a 16-bit FLAC (`Stream.bit_depth` isn't reliable for this — tidalapi defaults missing `bitDepth` fields to 16, so a trust-the-stream check would spuriously reject valid HiRes whenever Tidal omits the field). A proper fix runs mutagen post-download to inspect the actual file, then decides whether to delete + retry the next tier — a whole new failure mode with design trade-offs that deserve their own PR. The support logs don't show this happening. - The "manual remap still says Not Found" symptom. Might be downstream of this same bug (silent-AAC "success" hitting a later rejection), might be a separate task-state issue. Not guessing without logs from the retry path. - Quality-aware stub threshold. 100KB is a reasonable floor for real stub/preview detection and there's no evidence the universal threshold is misfiring in the wild. Field-verified status: desk-verified via unit tests and empirical checks against a live tidalapi import (confirming the `Quality` enum's str-subclass behavior). Not yet smoke-tested end-to-end against a real Tidal account with a HiRes-only-no-fallback setting — Netti93 or anyone else with that config should notice either the fix working (non-HiRes tracks fail honestly with a clear log line) or any regression before wider release. Files: - core/tidal_download_client.py — new `_verify_stream_tier` helper and `_QUALITY_RANK` table at module scope, called in the download loop after the stream is fetched and before any bandwidth is spent. Removed the old inline codec-based warning since the new check supersedes it. - tests/test_tidal_stream_tier_verification.py — ten tests covering match / upgrade / downgrade / unknown / defensive paths. - tests/test_tidal_search_shortening.py — fake `Quality` values brought in line with tidalapi's real values so both files share a consistent stub regardless of pytest collection order. - webui/static/helper.js — WHATS_NEW entry under 2.40 describing the rank-based tier comparison. Reported on Discord by Netti93 — the "same account works via Tidarr" comparison narrowed the cause to SoulSync's download path rather than an account/region issue.	1 month ago
Broque Thomas	a60546929e	Fix Album Completeness job reporting zero findings for every album ... Reported by sassmastawillis: the Album Completeness maintenance job scans 3127 albums in 0.1 seconds and reports 0 findings — for every user, regardless of whether their library is actually complete. Restoring an older DB surfaced 7 correct findings, so the code logic works; the DB state is what's making everything look complete. Root cause: `albums.track_count` is only ever written by server-sync paths — Plex's `leafCount`/`childCount` and SoulSync standalone's `len(tracks)`. It's the OBSERVED count of tracks SoulSync has indexed, which is always exactly what `COUNT(tracks)` returns for that album. The completeness job treated it as the EXPECTED total and compared it against the observed count. They're equal by construction, so `actual >= expected` is always true: skip, 0.1s scan, 0 findings. Fix: new `api_track_count INTEGER` column on `albums`, written only by metadata-source code paths. Populated in two places so the scan is fast and the fallback is robust. 1. Enrichment workers — shared helper `set_album_api_track_count` in `core/worker_utils.py`. Called by each worker's existing `_update_album` method alongside its other album-column UPDATEs: - spotify_worker: `album_obj.total_tracks` from the Spotify Album dataclass (already in hand, zero new API calls) - itunes_worker: same, from the iTunes Album dataclass - deezer_worker: `nb_tracks` from full_data, falling back to search_data when the full lookup didn't run - discogs_worker: count of tracklist rows where `type_=='track'` (Discogs tracklists interleave heading and index rows that shouldn't count as songs) Helper skips the write on zero/None/negative/non-numeric inputs so a source lacking track info can't clobber a good value a different source already wrote. Caller owns the transaction — helper just queues an UPDATE on the caller's cursor without committing, so it batches cleanly with each worker's existing multi-UPDATE pattern. Hydrabase worker deliberately not touched — it's a P2P mirror that doesn't write album metadata to the local DB. Hydrabase- primary users hit the fallback path below. 2. Album Completeness repair job — new `al.api_track_count` column in the SELECT, read first in the scan loop. On miss (album never enriched, or enrichment workers haven't run yet on a fresh install), falls through to the existing `_get_expected_total()` API lookup and persists the result via the same shared helper (wrapped in connection/commit management since the repair job runs outside a worker's batched transaction). Also removed `al.track_count` from the scan's SELECT — now unused since the observed count was the whole source of this bug, and leaving a dead SELECT would invite a future engineer to re-introduce the same comparison. Help text on the job card was reworded so it honestly describes current behavior ("counts cached during normal enrichment are used when available; otherwise the job queries a metadata source directly") rather than the old "active provider first, then others as fallback" phrasing, which doesn't match how the cache actually fills — any enrichment worker that runs can populate it, and the last writer wins. Document-only follow-up if this edge case ever bites in practice: add a `api_track_count_source` column so the scan can prefer the configured primary source's count over others (e.g. deluxe vs. standard edition mismatches). Not worth the complexity today. For existing users, the first completeness scan after upgrade is fast to the extent their library is already enriched: the workers already ran and populated `api_track_count` on their normal schedule. For brand-new installs, the scan's fallback path handles the cold start — slower, but correct, and subsequent scans are fast. Does NOT affect: - Download / post-processing / wishlist / sync code paths — none of them read `track_count` for completeness semantics. - Plex / Jellyfin / Navidrome / standalone sync — still write `track_count` exactly as before; `api_track_count` is a separate column they never touch. - Other repair jobs. - Any UI path — same finding schema, just correct counts now. Files: - database/music_database.py — idempotent migration adding `api_track_count INTEGER DEFAULT NULL` to the existing album-column check block. - core/worker_utils.py — new `set_album_api_track_count` helper with the documented skip-on-bad-input contract. - core/spotify_worker.py, itunes_worker.py, deezer_worker.py, discogs_worker.py — one-liner call from each `_update_album`. - core/repair_jobs/album_completeness.py — scan uses the cache; fallback path persists API-lookup results via the shared helper; help text updated to match actual behavior. - tests/test_worker_utils_album_track_count.py — 9 tests covering the helper's write/skip contract + no-commit invariant. - tests/test_album_completeness_job.py — 2 tests for the repair job's fallback-path wrapper. - webui/static/helper.js — WHATS_NEW entry. Credit: sassmastawillis spotted the bug; the "restored older DB finds 7 albums" signal pinpointed DB state over code logic and made the diagnosis tractable.	1 month ago
Broque Thomas	c454b1ebaf	MusicBrainz: Dedupe same-named homonyms in artist search results ... Typing "michael jackson" returned 7 identical-looking cards because MusicBrainz has many different PEOPLE sharing a canonical name — the King of Pop plus a NZ poet, a photographer, a mashup artist, a didgeridoo player, and more, all scoring 80+ on exact-name match. All 7 passed the score filter. All 7 rendered with the same fallback image because iTunes/Deezer only know the famous one. Fix dedupes by normalized (lowercase, whitespace-trimmed) name before building Artist dataclasses. Keeps the highest-scoring entry per name, so the King of Pop (score 100) wins over the others (all score 80-81). Artists with genuinely different names stay separate — a search for "the beatles" still surfaces tribute bands if they're above threshold. Implementation note: fetch `max(limit*3, 10)` from MB instead of `limit` directly, so the dedup pool is large enough to still return `limit` distinct artists after collapsing duplicates. Previously the raw fetch was capped at the caller's limit, which would have left fewer-than-requested results after dedup for common names. 3 new tests (49 total): - Dedupe collapses 5 same-named entries to 1 (keeps highest score). - Dedup key is case-insensitive and whitespace-normalized. - Dedup preserves distinct names ("The Beatles" vs "The Beatles Revival" stay separate). Live-verified: "michael jackson" now returns 1 card, "kendrick lamar" returns 1 card. Credit: kettui spotted duplicate Michael Jackson cards in the search UI.	1 month ago
Broque Thomas	b3722449fc	MusicBrainz: Fix artist images, total_tracks off-by-one, and Artist+Title queries ... Three bugs from kettui's follow-up review pass on the MusicBrainz search PR, all fixed in one commit because they share UI context. 1. Missing artist images on MB artist results MusicBrainz doesn't store artist images directly. My earlier commit returned `image_url=None` on every artist result and trusted the frontend's lazy-loader — but the lazy-loader's `/api/artist/<id>/image? source=musicbrainz` endpoint had no handler for MusicBrainz, so it silently returned None and the emoji placeholder stayed. Fix plumbs the artist name through: - `renderCompactSection` stashes `data-artist-name` on artist cards. - `search.js` and `downloads.js` lazy-loaders pass `name=<artist>` as a query param. - `/api/artist/<id>/image` accepts an optional `name` param. - `metadata_service.get_artist_image_url` has a new `musicbrainz` branch: since MB has no artist art, it searches fallback sources (iTunes/Deezer by configured priority) for the artist name and returns the first image found. Verified live — Metallica/Kendrick Lamar/Daft Punk all resolve to Deezer artist images via the name lookup. 2. total_tracks off-by-one on tracks with a release `_recording_to_track` initialized `total_tracks = 1` and then summed media track-counts on top. For an 11-track album, it reported 12. An adapter-level regression introduced when the recording-projection helper was extracted during the main MB refactor. Fix: initialize at 0, sum normally. Standalone recordings with no release (can happen for uncredited remixes etc.) still report 1 via an explicit fallback — so the existing "single track" case isn't broken. 3. "Artist Album Title" queries buried specific albums in the discography list Bare-name queries like "The Beatles Abbey Road" used to resolve "The Beatles" as the artist and then browse their full discography — Abbey Road was buried alphabetically among 200+ releases instead of being the top result. Fix adds a title-hint extractor. When the query starts with the resolved artist name followed by more words, the trailing portion is treated as a title hint. Browse results are filtered to those whose release-group title contains the hint. If the filter matches nothing, falls back to text-search with the hint as the title (the "keep the old split-by-whitespace fallback" path kettui called for). If text- search also misses, shows the full discography rather than nothing. 10 new tests in tests/test_musicbrainz_search.py (46 total): - Title-hint extractor: basic match, case-insensitive, whitespace tolerance, bare-artist-no-hint, artist-not-prefix-no-hint, word- boundary required (no false splits on "Metallicasomething"). - Browse filtering by title hint. - Text-search fallback when the title hint matches nothing in browse. - Bare-artist queries return the full discography unfiltered. - total_tracks for single-release, multi-disc, and no-release cases.	1 month ago
Broque Thomas	7dfe1ae88d	MusicBrainz: Resolve release-group MBIDs to a release on album click ... Clicking a MusicBrainz album returned 404 because the browse-based search path now stores release-GROUP MBIDs in Album.id, but `get_album` still hit `/ws/2/release/<mbid>` directly. Release-group MBIDs don't resolve as release MBIDs — MB 404s. User log: GET /api/spotify/album/b88655ba...?source=musicbrainz → 404 Error fetching release b88655ba...: 404 Client Error The fix requires a two-step resolution for the new browse path: 1. Look up the release-group with `inc=releases+artist-credits` to get the list of releases inside (original + reissues + regional + promo editions). MB release-groups routinely hold 5-20 releases. 2. Pick a representative release: prefer Official status over Promo, prefer releases with a real tracklist over stubs, then earliest date. 3. Fetch that release's full tracklist via `get_release`. Two extra seconds at the 1-rps rate limit, but it's on click, not on search results rendering. Structure: - New `MusicBrainzClient.get_release_group(mbid, includes)` method. - New `_pick_representative_release(releases)` helper encapsulates the ranking logic. - Tracklist projection extracted into `_render_release_as_album` so both paths share the same shape construction. - `get_album` tries release-group first; falls back to direct release lookup when the MBID turns out to be a release from the text-search fallback path. - Canonical Album.id stays the release-group MBID so a re-fetch with the same URL hits the same code path idempotently. 3 new tests (now 33 total): - End-to-end release-group → release resolution with mocked client - Fallback to direct release lookup when rg lookup misses - Representative-release picker ranks correctly Verified against live API with the exact MBID that 404'd for the user (b88655ba... for DAMN. by Kendrick Lamar): now returns in 1.2s with the full 14-track listing (BLOOD., DNA., YAH., ELEMENT., FEEL., ...).	1 month ago
Broque Thomas	ddbcdfe73a	MusicBrainz: Filter live/compilation bootlegs + chronological sort ... Three related fixes to make album/track results look like a real artist discography instead of a firehose of fan-compiled bootlegs. 1. Drop 'compilation' from the release-group browse primary-type filter. MB's OR filter (`type=album|ep|single|compilation`) silently breaks when 'compilation' is included — Metallica drops from 1076 matches to 82 because `compilation` is a SECONDARY type on MB, not a primary type. The invalid value corrupts the filter for all types, not just itself. Now we request `type=album|ep|single` which returns the full 1076; actual compilations (primary=Album + secondary=[Compilation]) are filtered out by the studio-preference logic below. 2. Filter release-groups with non-studio secondary-types (Live/Compilation/Soundtrack/Remix/Demo/Mixtape/Interview/Audiobook/ Audio drama). For Metallica, the first 100 browse results are 12 studio albums + 83 live bootlegs + 5 compilations — without this filter the Albums section was dominated by 2019-2021 broadcast recordings. Falls back to the unfiltered list if filtering leaves the result set empty (covers live-only niche artists). 3. Sort chronologically ASC by first-release-date. Wikipedia-style discography ordering — debut album on top, then chronological. Previous DESC sort put the most recent release on top which, for prolific artists, meant 2020s material before their classics. Track side of the same fix: - Re-orders each recording's `releases` array to put studio releases first before `_recording_to_track` picks up the first release for album context. Without this, MB's arbitrary release order often buried the canonical studio album under random live bootlegs. - Filters out recordings that only exist on live/compilation release- groups (keeps the ones with at least one studio release). Falls back to the full set if the artist has no studio recordings at all. - Sorts recordings by earliest studio-release year ASC so classic tracks surface first. Smoke test against live MB API confirmed: - Artists: [Metallica score=100] - Albums: Kill 'Em All (1983) → Ride the Lightning → Master of Puppets → ...And Justice for All → Metallica (Black Album) → Load → Reload → St. Anger → Death Magnetic → Lulu (2011) - Tracks: real Metallica recordings (Killing Time, Nothing Else Matters, Creeping Death, etc.) — a few remastered demos still leak in where MB metadata quality is thin, but the bulk is correct. - Total latency: 3.5 seconds. 4 new tests covering the studio filter, live-only fallback, preferred release ordering, and live-only recording exclusion. Credit: kettui flagged the poor MB results during PR #371 review.	1 month ago
Broque Thomas	8523724b03	MusicBrainz: Switch track lookup from browse to arid: search ... The previous commit's `browse_artist_recordings` call passed `inc=releases+artist-credits` — but MusicBrainz's recording browse endpoint rejects `inc=releases` with HTTP 400. The adapter's error handler returned an empty list, so the Tracks section stayed empty even though the fix was supposed to populate it. Browse without release info is useless for our search UI (tracks would render with no album), so swap to the fielded Lucene search `arid:<mbid>` on the `/recording` endpoint. That's the canonical MB pattern for "find recordings by this artist WITH release context": - arid: search accepts the artist MBID and returns recordings with `releases` (release-group, date, media) embedded in each result. - One API call per lookup, same as browse would have been. Renamed the method to `search_recordings_by_artist_mbid` so the name matches its behaviour — it's a search, not a browse. Adapter updated to call the new name; tests updated to match. Verified against the live API: Metallica's MBID returns 5 recordings in ~1.8 seconds (vs the previous 400 error).	1 month ago
Broque Thomas	73df2951e5	MusicBrainz: Construct Cover Art URLs instead of HEAD-probing them ... Cover Art Archive URLs are deterministic from the MBID: a GET either 307-redirects to the image or returns 404. The previous adapter fired `requests.head(timeout=3)` per search result to probe for the image first. 10 results × 3s worst-case = up to 30s of blocking HEAD calls before a search returned. The probe was defensive overhead — the frontend already handles 404 via `<img onerror>` fallback. Building the URL deterministically and letting the browser load it lazily collapses the tail latency to the real MB API calls (artist-search + browse = ~3s at the 1-rps rate limit). Also prefer release-group scope over per-release scope when both are available — release-group covers every edition of an album, so the hit rate is noticeably higher than pinning to a specific regional release. Removes now-unused `self._art_cache` and the `requests` import.	1 month ago
Broque Thomas	d7e232e01c	MusicBrainz: Artist-first browse for albums + tracks, keep text fallback ... Bare name queries (typing 'metallica') now resolve to an artist MBID via the fuzzy search added in the previous commit, then BROWSE that artist's release-groups and recordings instead of text-searching release/recording titles. That's the only way to fix the core garbage-results issue: MB indexes release/recording titles, not artist names, so 'recording:metallica' matches random tracks literally titled 'Metallica' (all scoring 100). Structure: - `_split_structured_query` — detects 'Artist - Title' / 'Artist – Title' / 'Artist — Title' shapes. When present, text-search is correct (user gave an explicit title to match). - `_resolve_top_artist` — memoized per-instance lookup for the top-scoring artist MBID. Backend fires artists/albums/tracks searches in parallel against one shared client instance, and albums+tracks both need the same artist lookup. Cache + lock means one HTTP call instead of three. - `_release_group_to_album` / `_recording_to_track` — shared projection helpers between the browse and text paths so both paths return the same dataclass shape. Search flow per kind: - `search_albums('metallica')` → resolve top artist → browse release-groups with `type=album|ep|single|compilation` → sort by type priority then release date desc → Album dataclasses for top N. - `search_tracks('metallica')` → resolve top artist → browse recordings with `inc=releases+artist-credits` → dedupe by normalized title (MB has many live/compilation variants of the same song) → sort by release date desc → Track dataclasses for top N. - `search_albums('foo - bar')` → structured query → text-search path (unchanged behavior, now score-filtered to 80+). - `search_tracks('foo - bar')` → same. - Both text-search paths also dedupe through `_search_albums_text` / `_search_tracks_text` helpers, which apply the 80-score filter that the artist-first path gets free from the resolver's threshold. Also dedupes text-path tracks through the new `_recording_to_track` helper, replacing ~60 lines of inline projection code. Net change is more lines overall (browse + helpers) but the text paths shrank and the garbage-results issue is fixed. Credit: kettui flagged the missing Artists section + unusable track results during PR #371 review.	1 month ago
Broque Thomas	434d1c382c	MusicBrainz: Re-enable real artist search (was returning empty) ... `MusicBrainzSearchClient.search_artists` has been a `return []` stub since the feature landed, with a comment claiming the MB tab 'doesn't show artists.' That's why kettui saw a missing Artists section on the search page — not a missing render, a hardcoded empty list. Re-enable it properly: - New `strict=False` parameter on `MusicBrainzClient.search_artist` sends a bare Lucene query instead of `artist:"..."`. MusicBrainz matches bare queries against alias+artist+sortname indexes together, which is the right behavior for user-facing fuzzy search (finds typos, aliases, sortname variants). `strict=True` remains the default for enrichment/AcoustID callers that want exact matches. - Adapter filters results to `score >= 80`. MB assigns a 0-100 Lucene score on every hit; the true artist + close variants score 100, tribute bands and lookalikes typically land in the 40-65 range. The cutoff keeps "Metallica" (100) and drops "Black Metallica Tribute Band" (60) without hand-curated lists. - Results returned as the same `Artist` dataclass used elsewhere in the search-tab adapter layer. `popularity` carries the MB score (0-100) so the frontend can sort/highlight top matches if desired.	1 month ago
Broque Thomas	e2a5a38cd2	MusicBrainz: Add browse endpoints for release-groups + recordings ... Add `browse_artist_release_groups(mbid)` and `browse_artist_recordings(mbid)` to MusicBrainzClient. These hit `/ws/2/release-group?artist=<mbid>` and `/ws/2/recording?artist=<mbid>` respectively — the correct MusicBrainz pattern for "give me everything linked to this artist." Why this matters: our current search adapter calls text-search (`release?query=...` / `recording?query=...`) for albums and tracks, which matches entity titles literally. Typing "metallica" hits unrelated releases titled "Metallica" and recordings named "Metallica" by obscure bands — every garbage match scores 100 because they're all exact title matches on the wrong field. Browse walks the artist→release-group and artist→recording links directly. Once we know the artist's MBID (from `search_artist`), browse returns their actual discography instead of title collisions. No behavior change yet — search adapter still uses the old path. Follow- up commit wires the new endpoints in. Reference: https://musicbrainz.org/doc/MusicBrainz_API — "Browse queries retrieve entities linked to a known entity" vs search.	1 month ago
Broque Thomas	3c48508c3f	MusicBrainz: Add project URL to User-Agent per API requirements ... MusicBrainz mandates a meaningful User-Agent with contact info, warning that bare strings can trigger IP blocking under load. Our client was sending `SoulSync/2.3` with no contact — and the search adapter passed an app version hard-coded at "2.3" that's now stale (UI is at 2.40). Fix: default contact to the project URL (`https://github.com/Nezreka/SoulSync`) when no email is supplied, so every request lands as `SoulSync/<version> ( https://github.com/Nezreka/SoulSync )`. Drop the search-adapter version suffix to a generic "2" since the exact UI minor version would add noise to every MB request without helping operators track issues. Reference: https://musicbrainz.org/doc/MusicBrainz_API — "it is important that your application sets a proper User-Agent string."	1 month ago
Broque Thomas	14893c85a9	Extract _build_source_only_artist_detail into core/artist_source_detail.py ... JohnBaumb's review: "If we're going to refactor the web_server.py soon, might as well start moving stuff away from web_server.py in our PRs. _build_source_only_artist_detail, make it a module, it's perfect." This continues the pattern the prior commit started with the source-ID lookup helpers: move the pure data-building logic to a side-effect-free core module, leave a thin wrapper in web_server.py that bridges the Flask response and the module-global clients. **core/artist_source_detail.py** — pure function that takes the artist id, name, and source plus dependency-injected per-source clients (spotify, deezer, itunes, discogs) and a Last.fm API key. Returns (payload_dict, http_status) so it isn't coupled to Flask. **web_server.py wrapper** — builds the client bag from the module globals (checks Spotify auth, constructs the Discogs client from the configured token, reads the Last.fm API key) and wraps the core return in jsonify. 147 lines of logic go away from web_server.py; the 24-line wrapper is purely glue. **tests/test_artist_source_detail.py** — 21 focused tests covering the response envelope, the source-specific ID-field stamping for all six supported sources, the dedup_variants=False contract (the behaviour that originally motivated the split of MetadataLookupOptions), per-source genre/follower extraction with safe handling of missing or throwing clients, and the Last.fm enrichment branch including the no-key and error-path cases. Runtime 0.26s.	1 month ago
Broque Thomas	02f26bf338	Make ApiCallTracker.save() atomic to prevent corrupt history files ... Cin observed that database/api_call_history.json was occasionally landing on disk truncated mid-write — `_load()` would log `History file is not valid JSON, starting fresh` and 24h of metrics would be lost. Root cause: `save()` opened the file in 'w' mode (which truncates to 0 bytes immediately) and then streamed JSON via `json.dump`. Any SIGINT/SIGTERM/crash between truncate and final write left the file half-formed — exactly Cin's symptom of the JSON cutting off mid-array. Switch to the standard atomic pattern: write to a sibling .tmp file, flush + fsync, then `os.replace` (atomic on every platform we run on). Failed writes also clean up the leftover .tmp file. The canonical file is now either the previous good copy or the new good copy — never a partial one.	1 month ago
Broque Thomas	e66af77ff6	Make artist_name Optional in find_library_artist_for_source ... Cin's review note: typing artist_name as plain `str` forced callers that didn't have a name to pass `""` as a placeholder, which leaks the parameter's emptiness contract into every call site and reads badly in tests. Switching to `Optional[str] = None` lets callers omit it. The function body's `if artist_name and active_server:` check already handles None and "" identically, so no body changes were needed. Tests that previously passed `artist_name=""` drop the argument; one new test covers the omitted-arg path explicitly. The web_server.py wrapper takes the same default for symmetry.	1 month ago
Broque Thomas	a097cf3d5a	Extract source-artist lookup helpers from web_server.py to core module ... Cin pointed out that the prior version of test_artist_source_lookup.py AST-parsed web_server.py to verify a constant and to string-match a function's response keys. That was a workaround for the fact that web_server.py can't be imported at test time (it boots Spotify, Soulseek, Plex, etc.) — the right answer is to move the logic into a side-effect-free module so it can be imported and tested directly. This commit: - adds core/artist_source_lookup.py containing the SOURCE_ID_FIELD map, the SOURCE_ONLY_ARTIST_SOURCES set, and find_library_artist_for_source - replaces the inline definitions in web_server.py with imports + a thin wrapper that injects the active media server - rewrites the tests to import from the core module directly: * mapping correctness is now a plain equality assertion * lookup behaviour is exercised against a real MusicDatabase * the AST parse and the string-matching contract test class are gone - drops the _build_source_only_artist_detail contract test entirely (the weakest of the four — it was just string-matching the function body); when that function moves to core/ it can get a real behavioural test alongside. Test runtime drops from ~161s to ~5.8s. All 18 tests pass.	1 month ago
Broque Thomas	f936b8cb12	Enrich source-only artist-detail response and skip discography dedup for source artists ... Source artists landing on /artist-detail were rendering an almost-blank hero — image + name + a tiny Download button — because the backend response only had {id, name, image_url, server_source: null, genres: []}. The library.js renderers do their best with what they have, and that wasn't much. Backend changes (_build_source_only_artist_detail): - Set the source-specific ID field (deezer_id / spotify_artist_id / itunes_artist_id / discogs_id / soul_id / musicbrainz_id) on artist_info so the corresponding service badge renders on the hero. - Try the source's own get_artist_info / get_artist for genres + followers (Spotify always; Deezer/iTunes/Discogs when available). Spotify also fills image_url if metadata_service.get_artist_image_url came up empty. - Last.fm enrichment by artist name — bio + listeners + playcount + lastfm_url. Mirrors what library artists get from the cached enrichment workers but on demand for source artists. - All enrichment lookups are wrapped in try/except so a 500 from any one source doesn't break the whole response. Frontend (library.js populateArtistDetailPage): - Watchlist button now initialises for source artists too. Falls back to artist.id + artist.name when there's no canonical Spotify identity (which is the common case for non-library artists). Discography dedup opt-out: - Added dedup_variants flag to MetadataLookupOptions (default True so library artists are unchanged). Source-only path now passes dedup_variants=False so every "Deluxe Edition" / "Remastered" / "Anniversary" variant the source returns is shown — matches the inline /artists page behaviour the user was comparing against. Result: source artists' hero now shows badges + bio + listeners + playcount + watchlist button + genres in addition to image and name. Discography lists every release the source returns, not the deduped canonical view.	1 month ago
Broque Thomas	8f85b0c251	Fix silent wrong-artist track downloads (Maduk/Tom Walker bug) ... User reported searching "Maduk - Leave A Light On" on Tidal silently downloaded Tom Walker's completely different song of the same name, then embedded Maduk's metadata into Tom Walker's audio. Three layers of defense all failed permissively. Two of them are fixed here; the third (score formula weights) was left alone since these two together cover it. Layer 1 fix — candidate artist gate (web_server.py:27782) Old: `if _best_artist < 0.4 and confidence < 0.85: continue` New: `if _best_artist < 0.5 and confidence < 0.85: continue` SequenceMatcher returns exactly 0.400 for "maduk" vs "tom walker" (5-char vs 10-char strings with coincidental char matches), which slipped past the strict `< 0.4` check. The word-boundary containment check earlier in the function already short-circuits legitimate formatting variations to sim=1.0, so falling to SequenceMatcher means strings are genuinely different. 0.5 closes the fencepost AND gives a small safety buffer. Layer 3 fix — AcoustID verification (acoustid_verification.py:316) When title matches but artist doesn't AND expected artist isn't found anywhere in AcoustID's returned recordings: Old: always SKIP (let file through, assume cover/collab) New: FAIL if artist_sim < 0.3 (clear mismatch) SKIP if artist_sim >= 0.3 (ambiguous — cover/collab/formatting) The 0.3 cutoff catches hard mismatches like Maduk/Tom Walker (sim ~0.2) while preserving benefit-of-the-doubt for borderline artist formatting differences. Legitimate covers and collabs where the expected artist appears anywhere in AcoustID's recordings still PASS via the existing secondary-match loop above. Both fixes are defense-in-depth — either alone would have caught this bug. Together they close the pre-download AND post-download gaps. All 292 tests pass. Version bumped to 2.39 with changelog entries.	1 month ago
Broque Thomas	0d0bbf38c9	Add query-shortening retry + qualifier guard to Tidal search ... Tidal's search engine chokes on long queries with multiple qualifier words (remix credits, edit labels, bonus-disc markers). User reported case: "maduk transformations remixed fire away fred v remix" returns 0, but shortening to "maduk transformations remixed fire away" works. Behaviour change: - On a 0-result search, retry with progressively-shortened variants (capped at 5 total attempts, 100ms pause between). - Variants (in priority order): 1. strip trailing "(...)" / "[...]" 2. strip all parentheticals/brackets 3-5. drop last 1 / 2 / 3 tokens 6. keep first half of tokens (rounded up) - Dedupes so identical variants don't re-query. Safety — qualifier-aware filter: - Variant keywords (Live / Remix / Acoustic / Extended / Unplugged / Instrumental / Karaoke / etc.) are extracted from the original query using word-boundary match so "edit" doesn't match "edition" and "mix" doesn't match "remixed". - If the original query carries any qualifiers, fallback results MUST contain those qualifiers in their track names — otherwise a shortened query could silently downgrade "Song (Live)" to the studio "Song". - Tracks that fail the filter are dropped. If no variant produces qualifier-matching tracks, returns ([], []) — the same outcome as the original code, so no regression. Contract preservation: - Never raises to caller (outer try/except catches orchestration errors). - Returns ([], []) on any failure path, same as original. - Original-query successes take the same code path as before — no behavioural change for queries that already work. - Defensive guards for None/empty/non-string query (early return). Logging: - Preserves original warning/error/info messages for back-compat log scraping. - Adds fallback-success INFO log ("Tidal fallback query succeeded: ...") so successful retries are visible in production logs. - Adds qualifier-filter INFO/DEBUG logs with kept/total counts. - Per-attempt exception logs at DEBUG (not ERROR) to avoid noise when retries succeed. - Traceback preserved on final failure. Tests (16 regression tests in tests/test_tidal_search_shortening.py): - Skowl's reported query reaches his working variant within the cap. - Paren/bracket stripping priority. - Short queries produce no variants. - All variants unique (dedup guard). - Progressive token drops present for long queries. - Qualifier extraction is word-bounded (no "edit" in "edition"). - Qualifier extraction is case-insensitive. - Track name filter requires ALL qualifiers. - Empty-qualifier list passes every track (original-query behaviour). All 292 tests pass.	1 month ago
Broque Thomas	78fa83c8ac	Add $cdnum template variable for multi-disc filenames ... New smart template variable that emits "CD01" / "CD02" etc. in filenames on multi-disc albums, and expands to empty string on single-disc albums so mixed libraries don't end up with "CD01" on every single. Template behaviour: - total_discs > 1 -> "CD{disc:02d}" (zero-padded, CD prefix) - total_discs <= 1 -> empty string - Both $cdnum and ${cdnum} bracket form supported - Empty value collapses cleanly via existing double-dash regex plus new leading-dash cleanup pass Wiring: - _apply_path_template in web_server.py (download pipeline) - _apply_path_template in core/repair_jobs/library_reorganize.py (Reorganize repair job) - total_discs added to every album-mode template context: * download pipeline album branch (uses resolved total_discs even for single-track downloads from search) * per-album Reorganize preview + apply endpoints (pre-scan all track tags once, take max disc_number) * Library Reorganize repair job (already had album_total_discs map, just added to context dict) Leading-dash cleanup added to _get_file_path_from_template (web_server) and _build_path_from_template (library_reorganize) so templates like "$cdnum - $track - $title" don't leave "- 05 - Title" on single-disc albums. UI: - Template hint in Settings -> File Organization documents $cdnum - Template validation variable list includes $cdnum - Reorganize modal variable reference shows $cdnum with example "CD01" Verified: - Multi-disc disc 1 -> "CD01 - 05 - Track" - Multi-disc disc 2 -> "CD02 - 05 - Track" - Single-disc -> "05 - Track" (no leading dash) - Templates without $cdnum behave unchanged - 276/276 tests pass	1 month ago
Broque Thomas	e5d4d61c0e	Fix watchlist content filters: live false positives + auto-scan bypass ... Two bugs reported in issue #320: 1. Auto-watchlist scan bypassed Global Override settings. scan_watchlist_profile applied _apply_global_watchlist_overrides, but the scheduled auto-scan called scan_watchlist_artists directly — bypassing the override. Users who unchecked "Albums" or "Live" under Watchlist → Global Override still saw full albums and live tracks added during nightly scans (per-artist defaults, which include everything, won). Moved override application into scan_watchlist_artists itself so every entry point respects it. scan_watchlist_profile now forwards the apply_global_overrides flag through to avoid double-application. 2. is_live_version (watchlist + discography backfill) and live_commentary_cleaner's content patterns used bare \blive\b, which matched verb uses like "What We Live For" by American Authors, "Live Forever" by Oasis, "Live and Let Die" by Wings. Tightened the live patterns to require clear recording context: (Live) / [Live Version] / - Live / Live at|from|in|on|version| session|recording|performance|album|show|tour|concert|edit|cut|take / In Concert / On Stage / Unplugged / Concert. Locked in 11 regression tests covering the reported false positives (What We Live For, Live Forever, Living on a Prayer, Live and Let Die) and the reported true positives (Dimension - Live at Big Day Out, MTV Unplugged, etc.). Version bumped to 2.37 with changelog entries.	1 month ago
Broque Thomas	457763cbab	Rebuild Discography Backfill: auto-wishlist, Fix All, section UI ... Root-cause fix for "scanning 50 artists" then silence: when the master repair worker was paused, force-run still kicked off _run_job but the job's first wait_if_paused() blocked forever because is_paused was tied to the master-enabled state. Force-run now bypasses master-pause — scheduled runs still respect it. Also fixes Fix All on discography findings doing nothing: the backend bulk_fix_findings query had a fixable_types allowlist that excluded missing_discography_track (and acoustid_mismatch). Added both. Backfill job rebuild: - auto_add_to_wishlist opt-in setting — creates findings AND pushes to wishlist during the scan - 3-option fix dialog (Add to Wishlist / Just Clear / Cancel) on single Fix, Bulk Fix selection, and Fix All (page-level) - Fix All "Just Clear" path uses the clear endpoint with job_id filter instead of the generic "may delete files" bulk-fix warning - Batched in-memory matching using get_candidate_albums_for_artist + get_candidate_tracks_for_albums (same fast path the Library pages use) - Rich album context per finding (id, name, album_type, release_date, images, artists, total_tracks) — flows through the wishlist pipeline so auto-processor classifies each track into the right cycle (albums vs singles) and post-processing gets correct folder/tags/art - Per-artist progress logs [N/50] Scanning ArtistName - Default interval 24h (was 168h); all release types default on; settings reordered with _section_* group headers (Core / Release Types / Content Filters) Repair settings UI: - Generic _section_<name> key convention renders as an uppercase group divider in the settings panel — any job can opt in - .repair-setting-row gets a dashed bottom border so label↔toggle pairing is visually clear - _prettifyRepairSettingKey fixes acronym capitalization (EPs, not Eps) Version bumped to 2.36 with changelog entries.	1 month ago
Broque Thomas	39a07e4bdf	Fix Discography Backfill silently skipping most releases ... Two bugs kept this job from finding anything useful on a typical library. 1. Wrong Deezer column name. The artists table has a deezer_id column (per music_database.py:1986), but the job looked for deezer_artist_id in both _scan_artist (line 132) and _get_library_artists (line 345). For Deezer-primary users, this meant the Deezer ID never made it into the source_ids map, so get_artist_discography fell back to artist- name-only search — slower and less accurate than an ID lookup. 2. Spotify-reported EPs were silently excluded. Spotify lumps EPs and true singles under album_type='single'. The previous _should_include_release short-circuited on album_type='single' and returned the include_singles setting (default False), so 4-6 track EPs on Spotify-primary libraries never survived the filter — even though include_eps defaulted to True. Only 7+ track full albums made it through. This is the main reason users felt the job did nothing. Fixes: - Use the correct deezer_id column name in both reference sites. - Restructure _should_include_release so only 'album', 'ep', and 'compilation' are trusted outright. Anything else (including 'single' and missing type) falls through to a track-count disambiguation matching the download pipeline's _get_album_type_display: 1-3 tracks = true single, 4-6 = EP, 7+ = album. A Spotify-returned 'single' with 5 tracks now correctly counts as an EP. Full suite stays at 263 passed. Ruff clean.	1 month ago
Broque Thomas	d9217237d2	Clean up 286 ruff lint errors to unblock CI and fix 10 latent bugs ... PR #340 added ruff to the build-and-test.yml CI gate, which surfaced 286 pre-existing lint errors. Left unfixed, every feature branch push fails CI. This commit resolves all of them so CI goes green and contributors can actually land work. Auto-fixes (248 of 286): removed unused f-string prefixes (F541), renamed unused loop control variables with underscore prefix (B007), removed duplicate imports (F811). Manually fixed 10 latent bugs ruff caught (all wrapped in try/except today, silently failing): - music_database.py: _add_discovery_tables() called undefined conn.commit() — would have crashed the iTunes-support migration for existing databases. Now uses cursor.connection.commit(). - web_server.py settings GET: referenced undefined download_orchestrator when it should be soulseek_client. Feature (_source_status on the settings payload) was silently missing for UI auto-disable logic. - web_server.py _process_wishlist_automatically: active_server undefined in track-ownership check. Auto-wishlist was falling through to the error handler and re-downloading owned tracks. - web_server.py start_wishlist_missing_downloads: same active_server bug in the manual wishlist path. - web_server.py _process_failed_tracks_to_wishlist_exact: emitted wishlist_item_added automation event with undefined artist_name and track. Automation event silently never fired correctly. - web_server.py discovery metadata enrichment: referenced cache without calling get_metadata_cache() first. Track enrichment from cached API responses was silently skipped. - web_server.py Beatport discovery worker: wing-it fallback branch used undefined successful_discoveries variable. Wing-it counter never incremented correctly. Now uses state['spotify_matches'] consistently with the rest of the function. - web_server.py _run_full_missing_tracks_process: stale import json mid-function shadowed the module-level import, making an earlier json.dumps() call reference an unbound local (F823). - web_server.py discovery loop: platform loop variable shadowed the module-level platform import (F402). - core/watchlist_scanner.py: 7 lambda captures of loop variables (B023 classic Python closure-in-loop bug) now bind at creation. No existing tests had to change. Full suite stays at 263 passed.	1 month ago
Antti Kettunen	6f79214439	Route artist image lookup through metadata service ... - Move /api/artist/<artist_id>/image resolution into core.metadata_service. - Resolve artist artwork through source priority, with explicit source/plugin overrides preserved. - Keep Spotify call tracking inside the client layer to avoid double counting. - Update similar-artist lazy loading to pass source context and add service coverage.	1 month ago
Antti Kettunen	b022a90997	Move MusicMap similar artist matching into metadata service ... - Relocate the streamed MusicMap similar-artist flow out of web_server.py and into core.metadata_service. - Match similar artists through the configured source-priority chain instead of assuming Spotify first. - Add iTunes artwork fallback so streamed artist payloads still carry image_url when search results are sparse. - Cover the new service behavior with tests.	1 month ago
BoulderBadgeDad	9863c947dc	Merge pull request #343 from kettui/fix/replace-print-with-logger ... Tidy up logging across the app	1 month ago
Broque Thomas	cf143f70af	Batch discography completion matching against pre-fetched candidates ... Artist detail pages ran check_album_exists_with_editions and check_track_exists per discography item, each firing 5+ title variations times 3 artist variations of fuzzy LIKE searches plus fallback broad-artist queries. For a 30-album artist that was ~450 SQL round-trips just to answer "which of these do I own." Hoist the artist's library albums and tracks into memory once per request via two new helpers — get_candidate_albums_for_artist and get_candidate_tracks_for_albums — and thread them through as optional candidate_albums / candidate_tracks params on check_album_exists_with_editions, check_album_exists_with_completeness, check_track_exists, check_album_completion, and check_single_completion. Batched path scores the same _calculate_album_confidence / _calculate_track_confidence against the in-memory list, preserving Smart Edition Matching and accuracy. Title-only cross-artist fallback still fires for collaborative-album edge cases. None on either param preserves legacy per-item SQL behavior for unaffected callers. Applied to both /api/library/completion-stream (library artist detail page) and iter_artist_discography_completion_events (Artists search page). Timing logs added to confirm the pre-fetch cost and loop elapsed time. On a Kendrick page load, per-album resolution drops from ~8 seconds to under the 50ms streaming sleep floor. Observed ~100x SQL reduction on the happy path.	1 month ago
Antti Kettunen	71e114b6fe	Tighten legacy logging output ... - collapse old multi-line debug bursts into single structured rows - remove leftover DEBUG-style prefixes from message text - keep the app log readable without losing useful trace detail	1 month ago
Antti Kettunen	01d118daa6	Separate AcoustID file logging ... - keep AcoustID logs out of app.log - route client and verification to logs/acoustid.log - align tag writer with the soulsync logger namespace	1 month ago