mirror of https://github.com/Nezreka/SoulSync.git
main
dev
video
fix/disable-beatport-features
johnbaumb-discover-redesign
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4.0
2.4.1
2.4.2
2.5.0
2.5.1
2.5.2
2.5.3
2.5.4
2.5.5
2.5.6
2.5.7
2.5.9
2.6.0
2.6.1
2.6.2
2.6.3
2.6.4
2.6.5
2.6.6
2.6.7
2.6.8
2.6.9
2.7.0
2.7.1
2.7.2
2.7.3
2.7.4
2.7.5
2.7.6
2.7.7
2.7.8
2.7.9
2.8.0
2.8.1
v0.65
${ noResults }
1308 Commits (ffbe669c67bd62e437fee82d3c49cfbe4e679001)
| Author | SHA1 | Message | Date |
|---|---|---|---|
BoulderBadgeDad
|
ca90c6ae6f |
Player revamp Phase 3a: extract stream state into testable per-session store
Foundation for multi-listener playback. Today web_server.py keeps ONE global
stream_state dict + one lock (web_server.py:747), so the whole server shares a
single 'currently playing' — every tab/device is a remote for the same
playback and two listeners collide. That global is woven through ~22 sites and
isn't unit-testable where it lives.
Lifted into core/streaming/state.py WITHOUT changing behavior:
- StreamSession: one playback's state, dict-compatible (s['k'], s.get,
s.update, 'k' in s) so existing call sites work unchanged, each with its
OWN RLock so distinct sessions never block/clobber each other.
- StreamStateStore: registry of named sessions; lazy + race-safe create;
DEFAULT session reproduces today's exact single-global behavior. Also
drop()/active_ids()/session_ids() for the eventual per-listener wiring.
web_server.py now binds (DEFAULT) and
. Drop-in: every .update()/[k]/.get()/ site behaves identically. _set_stream_state routes a reassign
through session.replace() so the store's session stays the live object (it's
effectively dead — prepare.py only mutates in place — but safe now).
Honest scope: this is the PROVABLE half of Phase 3. The remaining half (3b:
derive a per-browser session id, per-session Stream/ staging, executor
concurrency, disconnect cleanup) is browser-coupled and can't be verified
without driving 2+ live clients — deferred to a live session. The store API is
already shaped for it.
Tests (tests/streaming/, 33 total):
- test_stream_state_store.py (19): session dict-compat, isolation, lazy
create, drop rules, active_ids, concurrent-create race safety.
- test_stream_state_callsite_compat.py (7): every real web_server access
pattern (library/play, stream/start, status, audio guard, stop, prepare
in-place mutation, set->replace) against the exact object web_server binds.
- test_prepare.py +1: real prepare worker drives an actual StreamSession.
76 streaming+radio tests green; ruff clean; web_server.py parses.
|
1 month ago |
|
BoulderBadgeDad
|
472ec7ea01 |
Bump version to 2.6.5 (dev — prep for next cycle)
|
1 month ago |
|
BoulderBadgeDad
|
aa2806180e |
Fix: Soulseek album poll hangs on a stalled peer; failed batches never cleared
Two related bugs from the Slipknot album never finishing. 1) _poll_album_bundle_downloads hung when the peer stalled. The finish check needs every transfer terminal (completed/failed); the #715 grace only covers 'slskd says Completed but file not on disk'. A transfer stuck InProgress / Queued, or dropped by slskd, is none of those — so it blocked both the finish AND the grace exit, and the poll spun to the full ~6h timeout. Add a bundle-level stall guard: track a progress marker (#terminal transfers, total bytes across pending). If NOTHING advances for _stall_grace (180s) — no terminal transition AND no pending byte movement — the peer has stalled; mark the stuck transfers failed so the existing finish/all-failed checks resolve the bundle with whatever completed (missing tracks then fall back to the per-track matcher). Conservative: only trips when EVERYTHING is frozen, so a slow-but-progressing or still-queued transfer is unaffected. 2) Failed batches lingered in the UI forever ('No tracks loaded'). The auto-cleanup gate removed only complete/error/cancelled phases — 'failed' (e.g. an album-bundle hard failure) was missing, so it never aged out. Add 'failed' to the terminal set so it's removed after 5 minutes like the others. Tests (tests/test_soulseek_album_poll_stall.py): stalled peer → gives up with the completed subset (not the deadline); progressing bundle not falsely stalled; all-stalled → empty; dropped transfers stall out; clean finish unaffected. 124 download/soulseek tests pass; ruff clean. |
1 month ago |
|
BoulderBadgeDad
|
d3c897fb9d |
Wishlist: route the manual flow through the shared engine (manual == auto)
Stage 2: the manual 'Download Wishlist' flow now calls the same _run_wishlist_cycle engine the auto timer uses, so a manual scan runs the exact same code path as an auto scan. The old bespoke manual orchestration (build payloads + SERIAL inline dispatch) is deleted — its grouping/dispatch was a near-duplicate of auto's that had already drifted. Behavior changes (all intended, discussed): - Manual now dispatches album bundles in PARALLEL (album pool) like auto, instead of serially on one thread. A single cycle='albums' engine call covers the whole selection (albums bundled, singles/ungroupable -> per-track residual), so no 'both cycles' pass is needed. - The manual placeholder batch_id is reused as the engine's first sub-batch (first_batch_id), so the modal's existing poll target stays valid. - WishlistManualDownloadRuntime gains album_bundle_executor (wired in web_server, falls back to the shared pool when unset). - 'Don't start manual while auto is running' is unchanged — the existing route guard (is_wishlist_actually_processing -> 409) already covers it; no queue added. NOT touched: process_wishlist_automatically's behavior (proven by test_automation staying green in Stage 1) and the per-track download mechanics. test_manual_download.py rewritten to characterize the new behavior (engine dispatch via the executor, parallel, placeholder reuse, album-context). Full wishlist suite green (131); wishlist + automation = 392 passed. |
1 month ago |
|
BoulderBadgeDad
|
0898014364 |
Fix #740: run wishlist album-bundle downloads on a dedicated pool
A 2.6.3 change (
|
1 month ago |
|
BoulderBadgeDad
|
4fcc461616 |
Source IDs: add canonical registry, adopt at the highest-value sites
The same provider ID is stored under inconsistent column names across tables (deezer_id vs deezer_artist_id vs album_deezer_id vs similar_artist_deezer_id; spotify/itunes keep an entity qualifier, others don't; musicbrainz uses three nouns), so code checks 2-5 name variants everywhere. Add core/source_ids.py as the single source of truth for (provider, entity) -> column, with accessors that read an ID from a dict/sqlite3.Row robustly (canonical column first, then known aliases). NO database columns are renamed — these are the real names today; the registry just centralizes the knowledge. Targeted adoption (behavior-identical, verified): - core/artist_source_lookup.SOURCE_ID_FIELD now derives from the registry instead of duplicating the mapping (values unchanged). - web_server.py artist-detail builds artist_source_ids via source_id_map(...) instead of a hand-rolled per-source .get() dict. Broader call-site adoption deferred as clearly-scoped follow-up. Tests: tests/test_source_ids_registry.py (canonical columns, alias fallback, canonical-preferred, sqlite3.Row, source_id_map, SOURCE_ID_FIELD unchanged). Existing artist_source_lookup + artist_full_detail suites still green. |
1 month ago |
|
BoulderBadgeDad
|
779d729a08 |
Fix: Spotify playlist sync shows only 100 tracks (#736)
Reported by @Yug1900: a Spotify playlist's overview shows the correct count (e.g. 203) but the sync modal lists only 100, and only 100 would sync. Root cause in /api/spotify/playlist/<id> (the Spotify-tab track fetch): owned playlists fetch + paginate fine (sp.next loops over all pages). But when the authenticated playlist_items call FAILS (intermittently, often a 403 on followed / not-owned playlists) it fell straight back to the public embed scraper, which is hard-capped at ~100 tracks and has no pagination — and the result was returned as if complete. The overview total is correct because it comes from the playlist *metadata* listing, which succeeds independently. Fixes (additive — the owned/working path is unchanged): - Retry the items fetch once before resorting to the embed scraper, so a transient failure no longer silently truncates a large playlist. - Guard against silent truncation: compare fetched count against the playlist's known total; if short (or via the capped embed fallback), log a warning and return `incomplete: true` + `expected_total` instead of presenting a partial list as complete. No behavior change when the fetch succeeds in full (incomplete=false, extra fields ignored by the frontend). Lets a future UI tweak surface "got 100 of 203 — retry" rather than silently dropping tracks. |
1 month ago |
|
BoulderBadgeDad
|
d5f6a14ba1 |
Discovery lift (10/N): save_*_bubble_snapshot -> shared helper
Final cluster: the four structurally-identical snapshot endpoints
(discover_downloads, artist_bubbles, search_bubbles, beatport_bubbles) ->
core.discovery.endpoints.save_bubble_snapshot(...), wired via
_save_source_bubble_snapshot. All four validate a payload key, persist via
db.save_bubble_snapshot(kind, items, profile_id=...), and return a count +
timestamp; they differ only by:
- payload_key ('downloads' for discover, 'bubbles' for the rest) + its
no_data_error message.
- snapshot_kind, success_noun, and the info/except log subject + noun
("downloads"/"artists"/"albums/tracks"/"charts").
get_database / get_current_profile_id injected; get_json (request.json) invoked
inside the try, preserving the original 400/500 behavior incl. traceback dump.
Tests: +5 (missing key 400, None body 400, happy path with kind/profile/count/
timestamp, discover_downloads variant, exception -> 500). Full discovery suite:
210 passed.
web_server.py: -98 lines.
|
1 month ago |
|
BoulderBadgeDad
|
4caf36deb1 |
Discovery lift (9/N): update_*_playlist_phase -> shared helper
Ninth cluster: update_<source>_playlist_phase for the five sources sharing the
identical validation + full-message response (Tidal, Deezer, Qobuz,
Spotify-Public, YouTube) -> core.discovery.endpoints.update_playlist_phase(...),
wired via _update_source_playlist_phase + the _PHASE_LIST/_PHASE_LIST_YT
constants.
Per-source params:
- valid_phases — YouTube additionally allows 'parsed'.
- apply_extra_fields — Deezer/Qobuz/Spotify-Public also persist
download_process_id / converted_spotify_playlist_id from the body; Tidal and
YouTube do NOT, so they pass False (kept strictly 1:1 — the generic won't
apply those keys for them even if a caller sent them).
- not_found_message / error_label; get_json invoked inside the try.
NOT folded in: iTunes-Link — uses data.get('phase') (no "Phase not provided"
400) and returns a no-message payload.
Tests: +7 (404, missing-phase 400, invalid 400, happy path with extra-fields
suppressed, extra-fields applied when enabled, YouTube 'parsed' allowed,
exception -> 500). Full discovery suite: 205 passed.
web_server.py: -123 lines.
|
1 month ago |
|
BoulderBadgeDad
|
50ebfbd82f |
Discovery lift (8/N): update_*_discovery_match -> shared helper
Eighth cluster, the heavyweights (~110 lines each). The fix-modal
update_<source>_discovery_match for the four sources with the identical
structure (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.update_discovery_match(...), wired via
_update_source_discovery_match. Applies the user-selected Spotify track to the
discovery result (status/artist/album/duration/spotify_data/match-count) and
writes the manual fix to the discovery cache.
Per-source pieces are params:
- source_log_label / error_label.
- original_track_key ('tidal_track' / 'deezer_track' / ...).
- original_artist_getter: Tidal handles string-or-object artists
(first_artist_str_or_obj); the rest assume strings (first_artist_plain).
- web_server helpers (join/extract artist, build_fix_modal_spotify_data,
cache-key, get_database, active-discovery-source) injected.
- get_json passed as a callable and invoked INSIDE the try, preserving the
original's "request.get_json() inside try" behavior (malformed body -> 500).
NOT folded in (genuinely divergent): iTunes-Link (saves spotify_data directly
via a different cache signature), YouTube (multi-key original_track fallback),
ListenBrainz (entirely different unmatch-capable structure, no cache write),
Beatport.
Tests: +9 (extractors; 400/404/400 guards; full happy path with result
mutation + duration formatting + match-count + cache-save args; no-increment
when already found; cache error swallowed; get_json raise -> 500). Full
discovery suite: 198 passed.
web_server.py: -400 lines.
|
1 month ago |
|
BoulderBadgeDad
|
17c9e9b7b9 |
Discovery lift (7/N): start_*_sync -> shared helper
Seventh cluster: start_<source>_sync for the five sources with the identical
flow (Tidal, Deezer, Qobuz, Spotify-Public, YouTube) ->
core.discovery.endpoints.start_sync(...), wired via _start_source_sync.
Validates phase, converts discovery results, seeds sync state, posts a
"... Sync Started" activity item, and submits to the sync executor. Per-source
pieces are params:
- sync_id_prefix (f"{prefix}_{key}"), not_found/not_ready messages, convert_fn.
- name/image accessors: Tidal reads an object (playlist_name_obj/
playlist_image_obj), the rest a dict (playlist_name_strict/playlist_image_dict).
- activity_label vs error_label DIFFER for Spotify-Public ("Spotify Link
Sync Started" activity, "Spotify Public" logs).
- submit_sync_task glue (_submit_sync_task) closes over sync_executor /
_run_sync_task / get_current_profile_id so the helper stays global-free.
NOT folded in: iTunes-Link (no final info log), ListenBrainz (submits the
task WITHOUT a playlist_image_url arg), Beatport (extra debug logging, chart).
Tests: +6 (404, not-ready 400, no-matches 400, full happy path with
state/sync-infra/submit/activity assertions, resync phases allowed,
exception -> 500). Full discovery suite: 189 passed.
web_server.py: -172 lines.
|
1 month ago |
|
BoulderBadgeDad
|
7b6615b65a |
Discovery lift (6/N): get_*_playlist_states -> shared helper
Sixth cluster: the bulk-hydration get_<source>_playlist_states endpoints for
the five sources that build the identical per-entry dict + {"states": [...]}
shape (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link) ->
core.discovery.endpoints.get_playlist_states(states, *, error_label,
info_log_label=None), wired via _get_source_playlist_states.
iTunes-Link is the only one of the five without the "Returning N stored ..."
info log, so info_log_label is optional (iTunes passes None to suppress it).
NOT folded in: the YouTube/ListenBrainz get_all_*_playlists endpoints. They
return {"playlists": [...]} (different key) with a different field set
(url / created_at / playlist, no discovery_results) and filter out
mirrored_/profile-scoped entries — genuinely divergent, kept as-is.
Tests: +4 (list build + last_accessed bump + exact shape, empty, optional ids
default None, missing-required-field -> 500). Full discovery suite: 183 passed.
web_server.py: -116 lines.
|
1 month ago |
|
BoulderBadgeDad
|
44b032b6c0 |
Discovery lift (5/N): reset_*_playlist -> shared helper
Fifth cluster: reset_<source>_playlist for the four sources with byte-
identical bodies (Tidal, Deezer, Qobuz, Spotify-Public) ->
core.discovery.endpoints.reset_playlist(states, key, *, label,
not_found_message), wired via _reset_source_playlist. Resets phase/status to
'fresh', clears discovery/sync fields, cancels any discovery_future, and
preserves the original playlist payload.
Left with their own bodies (genuinely divergent):
- YouTube: status -> 'parsed' (not 'fresh'), no download_process_id, logs the
playlist name, "reset to fresh state".
- ListenBrainz: status -> 'cached', logs playlist title, returns
{"success": True, "phase": "fresh"} (different payload), _lb_state_key.
- iTunes-Link: state.update(...), no info log, "iTunes Link reset to fresh
phase".
Tests: +4 (404, full clear + playlist preserved + future cancelled, no-future
path, exception -> 500). Full discovery suite: 179 passed.
web_server.py: -100 lines.
|
1 month ago |
|
BoulderBadgeDad
|
8a9ed677ab |
Discovery lift (4/N): get_*_discovery_status -> shared helper
Fourth cluster: get_<source>_discovery_status (all eight sources, Beatport
included) -> core.discovery.endpoints.get_discovery_status(states, key, *,
not_found_message, error_label), wired via _get_source_discovery_status.
Unlike sync-status, the discovery-status response shape is byte-identical
across every source (phase/status/progress/spotify_matches/spotify_total/
results/complete), so Beatport folds in here too. Only the 404 string
("... discovery not found" vs "... playlist not found" vs "Beatport chart
not found") and the except-log label vary. ListenBrainz key via _lb_state_key.
NOT touched this cluster: get_*_playlist_state (the sibling endpoints).
Those genuinely diverge per source — different id-key name (playlist_id /
url_hash / playlist_mbid), presence of url / created_at / download_process_id,
Tidal's playlist.__dict__ serialization, and YouTube's strict (non-.get)
field access. Folding them would need a flag pile that wouldn't be a clean
1:1, so they keep their own bodies.
Tests: +4 (404, full response + last_accessed bump, complete=False when not
'discovered', missing-field -> 500). Full discovery suite: 175 passed.
web_server.py: -155 lines.
|
1 month ago |
|
BoulderBadgeDad
|
aad1d2b8f0 |
Discovery lift (3/N): get_*_sync_status -> shared helper
Third cluster: the get_<source>_sync_status routes (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link, YouTube, ListenBrainz) -> core.discovery. endpoints.get_sync_status(...), wired via _get_source_sync_status glue. This cluster carries the real per-source quirks, all captured 1:1 as params: - not_found_message (iTunes-Link uses "iTunes Link not found"). - error_label vs activity_subject — these DIFFER for Spotify-Public: the activity feed says "Spotify Link playlist ..." while the except log says "Error getting Spotify Public sync status". - playlist-name accessor, three styles lifted verbatim as named helpers: playlist_name_attr_or_unknown (Tidal: object .name), playlist_name_strict (Deezer/Qobuz/Spotify-Public/iTunes: state['playlist']['name'], can raise), playlist_name_safe (YouTube/ListenBrainz: .get default). The strict getter preserves the original's behavior of raising -> 500 AFTER phase/sync_progress were already mutated. - ListenBrainz key via _lb_state_key (caller-resolved). Beatport stays separate (different payload: status not sync_status, sync_id, no lock, chart key). Tests: +9 (3 name accessors incl. raise/fallback semantics; status 404s, running-no-mutation, finished+activity, error+revert+activity, and strict- getter-missing -> 500 after partial mutation). Full discovery suite: 171 passed. web_server.py: -244 lines. |
1 month ago |
|
BoulderBadgeDad
|
2d76a7c061 |
Discovery lift (2/N): cancel_*_sync + delete_*_playlist -> shared helpers
Second cluster. Two more sets of byte-identical per-source bodies: cancel_<source>_sync (Tidal, Deezer, Qobuz, Spotify-Public, iTunes-Link, YouTube, ListenBrainz) -> core.discovery.endpoints.cancel_sync(states, key, *, label, not_found_message, sync_lock, sync_states, active_sync_workers). Returns (payload, status_code); a thin web_server glue (_cancel_source_sync) wires the sync-infra globals + jsonify. Caller passes the resolved key (ListenBrainz transforms via _lb_state_key) and the exact 404 string (iTunes-Link uses "iTunes Link not found"). delete_<source>_playlist (Tidal, Deezer, Qobuz, Spotify-Public) -> delete_playlist_state(states, key, *, label, not_found_message), wired via _delete_source_playlist. Intentionally left with their own bodies (genuinely divergent, not 1:1): - Beatport cancel (cancels a stored sync_future, no message, warning log). - iTunes-Link / YouTube / ListenBrainz / Beatport deletes (different success messages, info-log wording, playlist-name extraction, /remove route, chart key). Tests: +11 in tests/discovery/test_discovery_endpoints.py covering cancel (404, active-worker cancel + state revert, worker-absent, no-sync-in-progress, label in message, exception->500) and delete (404, future cancel + removal, no/falsy future, exception->500 leaves state). Full discovery suite: 162 passed. web_server.py: -216 lines. |
1 month ago |
|
BoulderBadgeDad
|
628395eda5 |
Discovery lift (1/N): convert_*_results_to_spotify_tracks -> shared helper
First cluster of the per-source playlist-discovery deduplication. The convert_<source>_results_to_spotify_tracks functions (Tidal, Deezer, Qobuz, Spotify-Public, YouTube, ListenBrainz) plus the already-generic _convert_link_results_to_spotify_tracks were byte-identical apart from the source label used in their log line. Lift the shared body into core/discovery/endpoints.py as convert_results_to_spotify_tracks(results, source_label); the 7 web_server functions become 1-line delegations (names/signatures unchanged, so all callers and behavior are identical — 1:1). Beatport is intentionally NOT folded in: its converter coerces artist objects to strings and emits a different track shape (source field, album dict), so it keeps its own implementation. Tests: tests/discovery/test_discovery_endpoints.py (12) pin both input shapes (manual spotify_data / auto spotify_track+found), optional track/disc numbers, falsy-0 omission, field defaults, skip-on-neither, order preservation, if/elif precedence, empty input. web_server.py: -209 lines. Full discovery suite: 151 passed. |
1 month ago |
Broque Thomas
|
7145368d42 |
Basic search: visual overhaul + per-source picker in hybrid mode
Two things in this commit. Functional download / matched-download behaviour is untouched — same JS handlers, same routes for the download actions, same album-expand interaction. VISUAL REDESIGN - Glass search-bar card with accent radial wash + focus ring + pill primary search button - Source chip row above the search bar (see below) - Always-visible compact filter pill row (Type / Format / Sort) — pills carry both ``bs-filter-pill`` (new visual) and ``filter-btn`` (legacy class for ``resetFilters`` + ``applyFiltersAndSort`` in wishlist-tools.js to keep working) - Accent-tinted status pill matching the dashboard / auto-sync look - Album result cards: glass card with accent left-edge stripe, 52px brand-tinted cover icon, chevron expand indicator, pill action buttons (Download / Matched Album), accent glow on hover - Track result cards: glass row with accent stripe, 44px icon, pill action buttons (Stream / Download / Matched Download) - Multi-disc separators inside expanded album track lists styled with the accent treatment - Responsive: action button columns stack vertically below 900px New CSS lives in a self-contained ``webui/static/basic-search-v2.css`` sheet linked from index.html. Selectors are scoped to ``#basic-search-section`` for any class that already exists in style.css (``.album-result-card``, ``.album-icon``, ``.track-*``, etc.); the new ``bs-*`` prefixed classes for the search bar / filters / source row / status are unscoped because they only exist in the new markup. ``!important`` is used on the card-level rules to defeat the original unscoped ``.album-result-card`` etc. rules in style.css that would otherwise leak heavyweight padding / box-shadow / 56px icon styles into the new design. Also removed ``overflow: hidden`` from the original ``.album-result-card`` and ``.track-result-card`` rules in style.css — those two classes only render in ``downloads.js`` basic search results (verified via grep, two render sites only), so the removal can't impact any other UI. SOURCE PICKER (hybrid mode) - New ``GET /api/search/sources`` endpoint returns the list of active sources from the orchestrator's chain (or the single active source in single-source mode). - Frontend renders a chip row above the search bar. Click a chip to target that source for the next search; the chip's brand accent fills. - In single-source mode the lone chip is rendered as a dashed- border label so the user always knows what they're searching but can't accidentally try to switch to sources that aren't configured. - ``/api/search`` accepts an optional ``source`` body param. When set, ``core/search/basic.py:run_basic_search`` resolves the client directly via ``orchestrator.client(source)`` and calls its ``.search()`` instead of going through the hybrid chain. - Backwards compatible: omitting ``source`` falls through to the original ``orchestrator.search()`` call exactly as before. Unknown source names also fall back to the default — typo protection. TESTS (5 new + 6 pre-existing = 11 total in test_search_basic.py) - source param routes to specific client, NOT orchestrator chain - no source param preserves original orchestrator-default behaviour - unknown source name falls back to orchestrator default - ``run_basic_soulseek_search`` backwards-compat alias preserved - source-targeted path serialises albums + tracks correctly 101 search-suite tests pass. |
1 month ago |
|
Broque Thomas
|
6d54203710 |
Bump version to 2.6.4
- _SOULSYNC_BASE_VERSION → 2.6.4 - helper.js: '2.6.4' unreleased → 'May 28, 2026 — 2.6.4 release' - .github/workflows/docker-publish.yml default version_tag → 2.6.4 - pr_description.md: rewrite for 2.6.4 with #721 as the headline patch, 2.6.3 fixes carried forward unchanged (2.6.3 was bumped on dev but never reached main / docker, so 2.6.4 is the first release to ship this batch) |
1 month ago |
|
Broque Thomas
|
a9608e1bcb |
Bump version to 2.6.3
- _SOULSYNC_BASE_VERSION → 2.6.3 - helper.js WHATS_NEW unreleased flag → 'May 27, 2026 — 2.6.3 release' Note: .github/workflows/docker-publish.yml default version_tag was also bumped to 2.6.3 locally, but .github is gitignored in this repo — workflow updates need to land via the GitHub UI separately. The workflow_dispatch input is overrideable at trigger time regardless of the default, so this isn't blocking. |
1 month ago |
|
Broque Thomas
|
5771c5ba77 |
Album-bundle staging: clean Soulseek copies + sweep orphans at startup
Two related leaks in ``storage/album_bundle_staging/<batch_id>/``:
1. **Soulseek bundle cleanup was excluded.** The per-batch cleanup
at the end of a bundle download gated on:
(album_bundle_source or '').lower() in ('torrent', 'usenet')
The comment justified it as "slskd keeps its own completed
folders" — but the Soulseek bundle path ALSO copies completed
files into the private staging dir (``soulseek_client.py:1599``,
``copy_audio_files_atomically(completed, Path(staging_dir))``)
for the per-track workers to claim. Those copies persisted
forever; long-running installs accumulated stale GB. Extended
the cleanup gate's allow-list to include ``soulseek`` so the
per-batch dir is removed on bundle completion — same code path
that already worked for torrent / usenet.
2. **No sweep for orphan dirs.** Any leftover ``<batch_id>``
subdir from a previous-session crash, an errored batch, or a
pre-fix Soulseek bundle stayed on disk forever. Added
``sweep_orphan_album_bundle_staging(staging_root, active_batch_ids)``
that runs ONCE at server startup, before any batch can register
a staging dir. Removes every ``<batch_id>``-shaped subdir
whose id isn't in the active set. Safe by construction:
- Only touches subdirs of the configured staging root.
- Name-shape check (``entry.name == _safe_batch_dirname(entry.name)``)
rejects hand-placed dirs like ``.git`` or stray docs.
- ``shutil.rmtree`` errors log + continue — sweep must not
crash app startup over a permission glitch.
- active_batch_ids normalised through ``_safe_batch_dirname``
so colon-bearing batch_ids match their on-disk form.
Wired into the web_server startup right after the stuck-flags
diagnostic so it fires before anything else touches batches.
Tests
- ``test_downloads_lifecycle.py`` gained one regression test
pinning that Soulseek bundles now have their staging dir
cleaned (sibling to the existing torrent test).
- ``test_album_bundle_staging_sweep.py`` (NEW, 11 tests)
covers: orphan removal with no actives, active dirs preserved,
special-char batch_id normalisation, no-op on missing /empty
/empty-string staging root, non-dir entries skipped, unsafe-
name dirs preserved (.git etc.), partial rmtree failure doesn't
abort the rest, listdir failure returns 0 cleanly, default
None active set, defensive against empty / None entries in
the active set.
488 downloads tests pass.
For users with an existing "clean up old files" automation pointed
at this dir: stop pointing it there if you want — the auto-cleanup
+ startup sweep cover it now. Or leave it as belt-and-suspenders
with a relaxed (1h+) mtime threshold so it can't race a mid-batch
download.
|
1 month ago |
|
Broque Thomas
|
f976a6da53 |
Fix: Soulseek album-bundle downloads stuck on "failed" after slskd
finished the release (#715) Symptom (user @pavelcreates / @IamGroot60 on 2.6.2): - Click Download on an album in the search modal - slskd starts + completes every track of the release - 22+ minutes after the last completed download, batch flips to "failed" with no clear log line explaining why - Per-track Soulseek downloads on the same machine were fine Root cause: ``core/soulseek_client._resolve_downloaded_album_file`` probed three hard-coded candidate paths to locate each downloaded file in the slskd download dir: candidates = [ download_path / remote_filename, download_path / basename, download_path / *normalized_path_parts, ] On the common slskd config ``directories.downloads.username = true`` slskd writes files at ``<download_dir>/<username>/<filename>`` — none of the three candidates carry a username segment, so the resolver returned None for every file even though the file was physically present in a subdir one level deeper. ``_poll_album _bundle_downloads`` saw 0 completed_paths, kept spinning, and hit the master deadline (~30 min) before bailing the batch. Why per-track worked: ``web_server._find_completed_file_robust`` already does a recursive walk-by-basename + path-confirm against the remote directory components, so any layout slskd writes ends up resolved. The bundle path didn't go through it. Fix - Lifted the robust finder into ``core/downloads/file_finder.py`` as a pure function ``find_completed_audio_file(download_dir, api_filename, transfer_dir=None) -> (path, location)``. Zero globals; recursive walk; handles slskd dedup suffix ``_<10+digit-timestamp>``, YouTube / Tidal ``id||title`` encoded filenames, the AcoustID-quarantine subdir skip, basename collisions disambiguated by remote-path components, and a fuzzy-basename fallback above 0.85. - ``_resolve_downloaded_album_file`` keeps the three-candidate fast path (cheap probe for the slskd-flat default) but now delegates to the new helper when none hit, instead of giving up. - ``_poll_album_bundle_downloads`` tracks "slskd reports Completed but local resolver returns None" per key. When every remaining key has been in that state past a 45-second grace window, the poll exits early with an explicit error pointing at the likely ``soulseek.download_path`` mismatch instead of silently spinning until the master deadline. - ``web_server._find_completed_file_robust`` becomes a thin delegate so both callers share one finder. Legacy inline impl kept as ``_find_completed_file_robust_legacy`` for reference; to be removed next release. - Fixed misleading ``"(0 tracks, quality=)"`` log on the preflight- reuse path — was reading attrs off a None ``picked`` object. Tests (17 new in tests/downloads/test_file_finder.py) - Flat slskd layout - Username-prefixed (the #715 case) - Full remote tree preserved - Deeply nested username + tree - File genuinely missing returns None - Basename collision disambiguated by remote dirs - Single basename match wins regardless of dirs - slskd dedup suffix match - Short ``_<digits>`` (year) not treated as dedup - AcoustID quarantine subdir skipped - YouTube / Tidal ``id||title`` encoded filenames - transfer_dir fallback - Both dirs miss → (None, None) - Non-audio files ignored - Empty api_filename - Fuzzy match on punctuation variant - Fuzzy rejects below threshold 475 downloads tests pass after the lift. |
1 month ago |
|
Broque Thomas
|
65d7756da2 |
Resolve pre-existing ruff lint errors blocking CI
Five pre-existing lint errors on dev baseline (all introduced May 25-26 before this branch was cut) were blocking CI on this PR. Cleared as courtesy fixes so the merge isn't gated on unrelated tech debt: - web_server.py:22613 — F811 duplicate `urlparse` import inside `_parse_itunes_link_url` (already imported at module top, line 20). Removed from the inline `from urllib.parse import parse_qs, urlparse`; kept `parse_qs` since that one is only used here. - core/listenbrainz_manager.py:746 — S110 silenced with `# noqa: S110 — best-effort lookup, delete proceeds either way`. Matches the existing project convention used in web_server.py:1693, core/watchlist/auto_scan.py:463, core/library_reorganize.py:548. - core/playlists/sources/listenbrainz.py:236 — B905 `zip()` without explicit `strict=`. Added `strict=False` — preserves existing behaviour where `matched` can legitimately be shorter than `match_indices` on partial discover failure. - core/playlists/sources/listenbrainz.py:273 — S110 silenced with `# noqa: S110 — caller falls back to last cached playlist on refresh failure`. - core/playlists/sources/soulsync_discovery.py:105 — S110 silenced with `# noqa: S110 — manager persists last_generation_error on failure; surface existing snapshot`. The existing multi-line comment that already explained the swallow was rolled into the noqa justification so the rule + reason live on one line. Ruff `python -m ruff check .` now passes; 664 discovery + metadata tests still pass. |
1 month ago |
|
Broque Thomas
|
8dbbf13c61 |
Branch cleanup: lift manual-match helpers, fix length-pref ordering, profile-scope view toggle
Self-review pass on the prior three commits — kettui-style cleanup
that should have landed first time.
**Length-preference sort ordering (real bug):**
The `search_tracks_with_artist` stable sort that promoted length-known
recordings ran in `core/musicbrainz_search.py`, but the MB endpoint in
`web_server.py:search_musicbrainz_tracks` runs `rerank_tracks` after
it — which re-sorts by relevance score and dropped the length-pref
ordering down to tiebreaker-only. For canonical-same-song MB duplicates
that all score identically the tiebreaker survived, but the
order-of-operations was wrong.
Moved into `rerank_tracks` itself via a new `prefer_known_duration`
flag. Sort key sits between relevance score and the stable-order
tiebreaker so relevance still wins (length only decides ties, never
overrides a higher-relevance match). The MB endpoint opts in via
`prefer_known_duration=True`; Spotify / iTunes / Deezer callers stay
on the default-off path since their search results always include
length. Pinned with three new `TestRerankTracks` cases:
ties-promote-length, relevance-still-wins, default-off-unchanged.
**Route logic lifted to `core/discovery/manual_match.py`:**
Two pieces lived as inline route logic in `web_server.py` — the
`derive_manual_match_provider` fallback chain (payload.source →
active source → 'spotify') used by `update_youtube_discovery_match`,
and the `is_drifted_for_redo` predicate (cached provider differs from
active AND not manual_match) used by `prepare_mirrored_discovery`.
Per kettui's "extract logic from web_server.py, don't AST-parse it"
standard, both helpers now live in `core/discovery/manual_match.py`
with 12 dedicated unit tests covering fallback resolution order,
non-dict payload defenses, manual_match exemption from drift,
absent-provider legacy default, and edge cases.
Side benefits from the lift:
- `match_source` now derived once before the cache-save try block
instead of being duplicated in try + except (the except block existed
only because the original used `match_source` later — pre-computing
killed the duplication).
- `prepare_mirrored_discovery`'s `has_cached` check now reuses
`is_drifted_for_redo` with inverted polarity instead of restating
the field whitelist inline, so a future schema change only has to
land in one place.
- The mirrored-DB persist block now gates on `matched_data is not None`
to avoid a pre-existing latent NameError if the cache-save block
raised before matched_data construction.
**Enhanced toggle localStorage key now profile-scoped:**
`soulsync-library-view-mode` was global — two admin profiles would
share one preference. Wrapped in `_libraryViewModeKey()` which appends
`:${currentProfile.id}` when a profile is loaded, falls back to the
unsuffixed key otherwise (preserves pre-multi-profile saved values).
Tests:
- 12 new in `tests/discovery/test_manual_match.py` pinning both helpers.
- 3 new in `tests/metadata/test_relevance.py` pinning the
`prefer_known_duration` semantics.
- `test_search_tracks_with_artist_prefers_results_with_known_length`
renamed to `_does_not_resort_by_length` since the sort moved out of
this method. 664 tests pass across discovery + metadata suites.
|
1 month ago |
|
Broque Thomas
|
39f582a690 |
Mirrored playlist: stop Playlist Pipeline from reverting manual Fix-popup matches
User reported that manually mapping a mirrored-playlist track via the
Fix popup (either by search or by pasting an MBID) worked end-to-end
once — match saved, library track downloaded — but the next Playlist
Pipeline run flipped the track back to "Provider Changed" and forced
them to re-do the manual map every cycle.
Three independent issues were combining to cause this:
1. Hardcoded `provider: 'spotify'` on manual-fix save
`update_youtube_discovery_match` (the endpoint the Fix popup posts
to, also used by mirrored playlists since the frontend routes
`platform === 'mirrored'` through the YouTube endpoint) always
stamped the cached match as Spotify-provided. The Fix-popup cascade
actually queries the user's primary metadata source first and falls
back to Spotify / Deezer / iTunes / MusicBrainz — so a user on
MusicBrainz primary picking an MB result still had it saved as
`provider: 'spotify'`. The next prepare-discovery call (which
compares cached_provider to the active source) then immediately
classified the match as drifted and pending re-discovery. Fixed by
deriving `match_source` from `spotify_track.get('source')` (every
*_search_tracks endpoint stamps `source` on results) with a fallback
to `_get_active_discovery_source()` for the MBID-paste path (which
uses the lean flat shape that doesn't carry source). `matched_data['source']`
and the mirrored `extra_data['provider']` both now use the derived
value. `match_source` is also recomputed in the cache-save except
handler so the downstream mirrored-DB save still has it.
2. Discovery worker re-queueing manual matches as "incomplete"
`run_playlist_discovery_worker` in `core/discovery/playlist.py`
re-adds any track to `undiscovered_tracks` when its `matched_data`
lacks `track_number` or `album.id` / `album.release_date`. The
check was designed as a legacy-fix backfill for old discoveries
that lost those fields to a Track-dataclass stripping bug. But
manual fixes from the popup are *intentionally* lean — search-
result rows don't include `track_number` (none of the search
endpoints return it), and the MBID-lookup flat shape doesn't
carry `album.id` / `release_date` (the recording lookup returns
only `album.name`). So every manual match looked "incomplete" and
got re-discovered every pipeline run, overwriting the user's pick
with whatever the auto-search ranked first. Manual matches now
short-circuit ahead of the incomplete-data branch.
3. `prepare_mirrored_discovery` ignored the `manual_match` flag
Independent of the provider-stamping fix above, the prepare-
discovery endpoint that powers the mirrored-playlist UI did its
own `cached_provider != current_provider` check and didn't honour
manual_match either. Defence in depth — even if a future code
path stamps the wrong provider on a manual match, the flag now
anchors it as cached. `has_cached` also extended so manual
matches with off-provider stamps still count toward the cached
tally for phase classification.
Tests:
- new `test_manual_match_skipped_even_when_matched_data_incomplete`
in `tests/discovery/test_discovery_playlist.py` pins the worker
short-circuit using a realistic MB-shape matched_data (album dict
without id / release_date, no top-level track_number). 16 existing
tests still green; 848 across discovery / metadata / automation
suites pass.
|
1 month ago |
|
Broque Thomas
|
cf5da04439 |
Roll LB Weekly / Top series into single rolling mirrors (Phase 1c.2.1)
ListenBrainz publishes "Weekly Jams for X" / "Weekly Exploration
for X" with a fresh MBID every week, and "Top Discoveries of YYYY
for X" / "Top Missed Recordings of YYYY for X" with a fresh MBID
every year. Auto-mirroring those per-period yielded one mirrored-
playlist row per week/year — useless for Auto-Sync schedules
because the underlying LB playlist never updates, only a brand new
playlist replaces it. The user accumulates 100+ dead Weekly Jams
rows per year if they discover regularly.
This commit collapses each family into a single ROLLING mirror
keyed by a synthetic ``source_playlist_id`` (e.g.
``lb_weekly_jams_Nezreka``). Each new period UPSERTs into the same
row, so the user gets one stable Auto-Sync schedule per series
that automatically picks up the latest period's tracks on every
refresh. Non-series LB playlists (user-created, collaborative,
Last.fm radios for a specific seed) continue to mirror under
their per-playlist MBID as before. Per-period LB playlists are
still visible + usable on the LB Sync tab — only the mirror layer
collapses.
- ``core/playlists/lb_series.py`` (new) — series-detect helper
with regex patterns + canonical-name + LIKE-pattern template
for each known LB family. Exposes
``detect_series(title)``, ``is_series_synthetic_id(id)``, and
``list_series_synthetic_ids()`` so both the JS auto-mirror hook
and the LB adapter can speak the same language.
- ``GET /api/listenbrainz/series-detect?title=...`` — thin HTTP
shim around ``detect_series`` so the auto-mirror JS doesn't
duplicate the regex.
- ``ListenBrainzPlaylistSource.get_playlist`` now recognizes
synthetic series ids — it queries the LB cache for the newest
cache row whose title matches the series' LIKE pattern and
resolves to that row's MBID before fetching tracks. The mirror's
meta keeps the synthetic id so refreshes always re-resolve to
the latest period.
- ``_mirrorListenBrainzAfterDiscovery`` (sync-services.js) calls
the new detect endpoint when discovery completes — if a match
comes back it swaps the per-period MBID for the synthetic id +
the canonical name. Existing Last.fm radio routing logic stays
intact (Last.fm radios aren't a series).
- ``ListenBrainzManager._cleanup_per_period_series_mirrors`` —
one-shot consolidation sweeper runs in ``_cleanup_old_playlists``
+ deletes any legacy per-period mirror rows so the consolidated
rolling mirror is the only one left. Idempotent — only matches
per-period titles ("Weekly Jams for ..., week of ...") and never
the canonical rolling-mirror titles ("ListenBrainz Weekly
Jams").
- 11 new tests pin the detector + synthetic-id helpers; 236 total
across adapter + automation + lb-series suites green.
|
1 month ago |
|
Broque Thomas
|
38e35930a9 |
Add Last.fm Radio tab to Sync page (Phase 1c.2)
Sibling to the ListenBrainz Sync tab from Phase 1c.1. Last.fm Radio
playlists already live in the same ``listenbrainz_playlists`` table
as LB ones (``playlist_type='lastfm_radio'``) and run through the
same MB-track discovery worker, so this tab is intentionally thin
— list + render + delegate. Card click hands straight off to the
LB Sync-tab click handler since the downstream modal + state
machine are identical.
- ``webui/index.html``: new ``<button data-tab="lastfm-sync">``
+ tab content container between the LB tab and the existing
Import / Mirrored tabs. Plus a ``<script>`` tag for the new
module.
- ``webui/static/sync-lastfm.js`` (new): ``loadLastfmSyncPlaylists``
hits the existing ``/api/discover/listenbrainz/lastfm-radio``
endpoint, ``renderLastfmSyncPlaylists`` mirrors the LB card
shape with a ``📻`` icon + a ``.lastfm-playlist-card`` brand
class, click handler forwards to
``handleListenBrainzSyncCardClick``.
- ``webui/static/sync-listenbrainz.js``: the shared 500ms refresh
loop now iterates LB + Last.fm cards in one pass and treats
either tab as "active" for liveness. No second loop needed.
- ``webui/static/sync-services.js``: new tab-activation branch in
``initializeSyncPage`` mirrors the LB pattern.
- ``webui/static/style.css``: ``.lastfm-icon`` SVG (Last.fm "as"
logo, red), and ``.lastfm-playlist-card`` joins the unified
card selector group with the Last.fm-red accent
(``rgba(213, 16, 7, ...)``).
- ``web_server.py``: the lastfm-radio endpoint now includes
``track_count`` in its JSPF payload (same fix as the LB
endpoints last commit).
- WHATS_NEW entry added under 2.6.3.
Mirrors created from Last.fm radios participate in the same auto-
trim Phase 1c.1's cascade-delete hook does — when the LB manager
rotates a stale ``lastfm_radio`` row out of its 5-most-recent
window, the matching ``source='lastfm'`` mirror row is removed
along with it. Library files stay on disk.
225 tests across adapter + automation suites still green; this
commit adds no Python paths to test.
|
1 month ago |
|
Broque Thomas
|
f521be7720 |
LB Sync tab: fix track counts + auto-mirror on discovery complete
Two follow-ups to the LB Sync tab work: 1. **Track counts all showed 0.** The ``/api/discover/listenbrainz/*`` endpoints assemble a JSPF-shaped payload but drop the cached ``track_count`` field from the underlying ``listenbrainz_playlists`` row — the JSON the frontend sees only carries ``title`` / ``creator`` / ``annotation`` / an empty ``track`` array. The Discover-page renderer worked around it by hard-coding a fallback of 50; the Sync-page renderer had no such fallback, so every card displayed "0 tracks". Backend now includes ``track_count`` directly in each playlist payload (it's already in the cached row) so any frontend can render an accurate count without resorting to a default. JS still falls back to ``annotation.track_count`` and then ``track.length`` for older callers. 2. **LB playlists never landed in Mirrored Playlists.** The existing ``/api/listenbrainz/sync/start/<mbid>`` endpoint runs the converted Spotify tracks through ``_run_sync_task`` — i.e. it pushes them to the user's media server (Plex / Jellyfin / Navidrome / SoulSync) as a server-side playlist. It does NOT call ``database.mirror_playlist``. So no ``mirrored_playlists`` row gets created and the playlist can't be picked up by the Auto-Sync scheduler, can't show up under the Mirrored tab, doesn't participate in pipeline automations — the whole point of the Sync-tab unification. Tidal works because Tidal mirrors on tab load with raw tracks then enriches via discovery. LB tracks only have provider IDs *after* discovery, so the equivalent moment for LB is "discovery complete". Added ``_mirrorListenBrainzAfterDiscovery(mbid)`` that pulls the matched ``spotify_data`` out of ``discovery_results`` and posts to ``/api/mirror-playlist`` via the existing ``mirrorPlaylist`` helper. Hooked into both the WebSocket and HTTP-poll completion handlers of ``startListenBrainzDiscoveryPolling``. UPSERT-keyed on (source, source_playlist_id, profile_id), so re-running discovery is a safe no-op refresh. Result: any LB playlist the user discovers (from either the Discover page or the new Sync tab) now lands in ``mirrored_playlists`` with ``source='listenbrainz'`` + matched tracks carrying canonical ``extra_data`` JSON, ready for the Auto-Sync refresh + sync pipeline wired up in Phase 1a + 1b. |
1 month ago |
|
Broque Thomas
|
246503066b |
Fold provider-matching into PlaylistSource contract (Phase 1b)
Adds ``discover_tracks(tracks) -> List[NormalizedTrack]`` to the PlaylistSource interface. Sources whose tracks already carry provider IDs (Spotify, Tidal, Qobuz, YouTube, Deezer, Spotify public, iTunes link, SoulSync Discovery) inherit a no-op default; ListenBrainz + Last.fm override to run the matching engine. This closes the last gap before LB / Last.fm / SoulSync Discovery can land as Sync-page mirror sources: the refresh handler now calls ``source.discover_tracks(...)`` whenever a source returns tracks with ``needs_discovery=True``, so mirrored LB rows arrive already discovered + ready for the sync pipeline. Previously, LB playlists ran through a separate state-machine worker tied to the Discover-page UI, with results stored in ``discovery_cache`` instead of ``mirrored_playlist_tracks.extra_data``. Changes: - ``core/playlists/sources/base.py`` — PlaylistSource switches from Protocol to ABC so a concrete default for ``discover_tracks`` can live on the base class. The four real-work methods stay ``@abstractmethod``; instantiating an adapter that forgets one fails loudly at construction. - ``core/discovery/matching.py`` (new) — pure ``match_mb_tracks`` helper that runs Strategy-1-only matching-engine queries against Spotify (primary) or iTunes (fallback). No state machine, no discovery-cache writes, no wing-it stub — that richer flow stays in ``core/discovery/listenbrainz.py`` for the Discover-page UI. - ``ListenBrainzPlaylistSource`` + ``LastFMPlaylistSource`` take an optional ``discover_callable`` constructor arg. Last.fm reuses the LB implementation since the track shape is identical. - ``bootstrap.build_playlist_source_registry`` accepts a ``discover_callable`` kwarg and wires it into LB + Last.fm adapters. - ``web_server.py`` boot constructs the discovery callable from the existing matching engine + ``_discovery_score_candidates`` + Spotify / iTunes clients, passes through to the registry. - ``refresh_mirrored.py`` adds a small ``_maybe_discover`` helper that calls ``source.discover_tracks(...)`` between fetch and ``to_mirror_track_dict`` projection — only fires when at least one track has ``needs_discovery=True``, so the normal Spotify / Tidal / etc. refresh path stays a zero-cost pass-through. Tests: - 5 new adapter tests: default no-op pass-through, LB discovery with mixed matches/misses, LB no-callable fallback, Last.fm shares the LB implementation, mirror-dict spotify_hint emit. - 1 new automation test: end-to-end LB refresh with a stub discover_callable proves the matched_data lands in ``mirror_playlist_tracks.extra_data`` after the registry refresh + discover hop. 225 tests across adapter + automation suites green. |
1 month ago |
|
Broque Thomas
|
8c41b05fe8 |
Refactor refresh_mirrored to use unified PlaylistSource registry
Phase 1a of the Discover-to-Sync unification. The mirrored-playlist refresh handler used to branch per-source through a ~190-line if/elif chain (Spotify, Spotify public, Deezer, Tidal, YouTube). Each branch hand-built its own ``extra_data`` JSON for the matched- data block. With every new source we considered for Sync-page mirror support (ListenBrainz, Last.fm radio, SoulSync Discovery, iTunes link), that chain would have grown a new elif. This commit lifts the per-source logic into the existing adapter layer and collapses the dispatch to a registry lookup: - ``core/playlists/sources/deezer.py`` — new adapter so the registry covers every source the refresh handler previously branched on. - ``core/playlists/sources/bootstrap.py`` — single helper that builds a populated registry from injected getter callables. Both ``web_server.py`` boot and the automation test fixtures call it, so the two construction paths can't drift. - ``core/playlists/sources/base.py`` — ``to_mirror_track_dict`` projection helper centralises the NormalizedTrack → DB-row conversion (including the discovered/matched_data and spotify_hint extra_data shapes the downstream sync + wishlist consumers already expect). - Spotify adapter now populates ``extra['discovered']`` + an ``extra['matched_data']`` block when fetching via the authed API, so Spotify mirrors keep landing pre-discovered (matches the pre-refactor contract pinned by ``test_spotify_refresh_writes_to_db``). - Spotify-public adapter populates ``extra['spotify_hint']`` so the discovery worker can skip its search step and jump straight to enrichment for the known track ID. - All artist-name fields now project to first-artist-only across every adapter — matches the pre-refactor mirror_playlist DB shape (``t.artists[0]``). ``refresh_mirrored.py`` shrinks ~190 → ~80 lines and keeps: - the file/beatport unrefreshable-source filter, - URL extraction from ``description`` via ``require_refresh_url`` for spotify_public + youtube, - the Spotify-public → authed-Spotify fallback when the user is signed in (handler-level branch, not in any adapter), - the Tidal-not-authenticated soft-skip log (skip, not error), - existing-extra_data preservation across refreshes, - the ``playlist_changed`` automation event emit on track-set delta. Test scaffolding: - ``_build_deps`` in ``tests/automation/test_handlers_playlist.py`` now builds a default registry from the passed clients via ``build_playlist_source_registry``, so existing refresh tests exercise the same path without per-test changes. New tests cover Tidal-not-authed soft-skip, Deezer refresh writes plain tracks, YouTube refresh reads URL from description, and Spotify-public uses authed Spotify when signed in. - 4 new adapter tests for Deezer projection + ``to_mirror_track_dict`` (minimal track, Spotify matched_data, Spotify-public spotify_hint). - ``playlist_source_registry`` field on ``AutomationDeps`` defaults to ``None`` so the other 5 automation test files (which don't exercise refresh_mirrored) keep working unchanged. 220 tests across automation + adapter suites green. |
1 month ago |
|
Broque Thomas
|
718eb0cb10 |
Add iTunes / Apple Music link import tab on Sync page
New iTunes Link tab between Deezer Link and YouTube. Accepts album, track, and playlist URLs from music.apple.com / iTunes. Pulls the tracklist, runs it through the same discovery -> sync -> download pipeline as the other link tabs. Apple Music playlists go through amp-api with a Bearer JWT scraped from the SPA. The legacy meta-tag and inline `"token":"..."` paths are gone in the current music.apple.com SPA, so the extractor now walks the page's `<script src>` list (prioritising index/chunk/main bundles), fetches up to 8 JS bundles, regex-matches JWT-shaped strings, and base64-decodes each payload to confirm it carries Apple media-api claims (`root_https_origin`, or `iss + iat + exp`) before trusting it. Filters out analytics / error-reporter JWTs that also ship in the bundle. Tokens are cached at module scope for 6h behind a threading.Lock so the three-worker discovery executor doesn't thunder-herd Apple on cold start, and amp-api calls go through a single helper that on 401 invalidates the cache, refetches the page, force-refreshes the token, and retries the request once. The playlist fetcher memoises the page HTML for the cache-miss path so we don't refetch it for every paginated `/tracks` page. spotify_public discovery worker accepts the new platform shape so iTunes Link reuses the same matching code path as Deezer Link and Spotify-public. UI bits live in the sync-services.js iTunes Link tab, with platform plumbing through wishlist-tools.js for the multi-source state map. |
1 month ago |
|
Broque Thomas
|
dad1b5109e |
Add _build_library_tag_db_data helper
Extract repeated DB tag payload construction into a new _build_library_tag_db_data(track_data, album_genres) helper and replace in multiple endpoints. The helper builds the metadata dict (title, artist_name, track_artist, album_title, year, genres, track/disc numbers, bpm, track_count, thumb_url) and populates artists_list by splitting track_artist on ';'. Added tests (tests/test_library_tag_payload.py) to verify artists_list creation, genre propagation, thumb_url selection, and fallback behavior when track_artist is missing. This reduces duplication and ensures consistent tag payloads across tag-preview, batch preview, and tag-writing flows. |
1 month ago |
|
Broque Thomas
|
26eeb1e9a1 |
Bump base version to 2.6.2
Update version references for the 2.6.2 release: change the workflow dispatch input description and default to 2.6.2 in .github/workflows/docker-publish.yml, and update the _SOULSYNC_BASE_VERSION constant in web_server.py to 2.6.2 so release metadata and build/version strings reflect the new patch release. |
1 month ago |
|
Broque Thomas
|
dfdc6c6277 |
Restyle Auto-Sync manager and fix loading regressions
Three problems wrapped into one pass on the Playlist Auto-Sync surface: 1. Visual: the manager modal had its own vibe (radial gradient, pill tabs, sky-blue chrome) that didn't line up with the rest of the app. Reworked the modal shell, KPI summary, live pipeline monitor, tab bar, schedule board sidebar, and column cards to use the standard SoulSync patterns — gradient `#1a1a1a → #121212`, accent-tinted 1px border, 20px radius, underline tabs, dense dark card pattern that Automations + Library pages already use. Modal now uses near-full screen so there's room for the schedule board without horizontal scroll pain. Run history cards followed the same path: slim horizontal row mirroring `.automation-card` plus an expanded detail that mirrors the Automations run-history modal (stats-grid + facts row + result pills + log section). 2. Hang: the previous SQL fix for the run-history "in library" count added `COLLATE NOCASE` on the join columns of `tracks` and `artists`. SQLite can't use `idx_artists_name` or `idx_tracks_title` when the comparison collation doesn't match the column collation, so the join did a full table scan per mirrored playlist track. ~18s per playlist × 30 playlists = `/api/mirrored-playlists` hung indefinitely and the modal stayed at "Loading schedule…" forever. Switched the join back to case-sensitive equality (~6ms per playlist, 3000× faster). Spotify names canonicalize to the same form as library imports so the recall loss is in the rounding error of pure case-only mismatches. 3. Slowness: even after the hang fix, each modal open spent ~1.5s gathering per-playlist status counts. The endpoint looped `get_mirrored_playlist_status_counts(playlist_id)` per row, which opened a fresh SQLite connection + PRAGMA setup each time. Added `get_all_mirrored_playlist_status_counts(profile_id)` which returns counts for every mirrored playlist owned by the active profile in 4 batched `GROUP BY` queries over a single connection. Modal load dropped to ~280ms. Also fixed: `tracks.artist` reference in `get_mirrored_playlist_status_counts` that never worked since the schema went relational — the query threw "no such column", got swallowed by the try/except, and the in-library count silently defaulted to 0 on every playlist. Rewired to join through `artists`. `get_mirrored_playlist_status_counts` (single-playlist) kept for callers that still want it, but the modal endpoint uses the batched version. |
1 month ago |
|
Broque Thomas
|
efdcde1892 |
Add playlist auto-sync run history
Persist per-playlist pipeline run snapshots from the shared playlist pipeline, expose a history API, and upgrade the Auto-Sync modal with live pipeline monitoring, Run now controls, and a runs-style history tab. |
1 month ago |
|
Broque Thomas
|
f83c671570 |
Add direct mirrored playlist pipeline runs
Expose playlist-native run and status endpoints that reuse the shared mirrored playlist pipeline engine while routing progress into playlist UI state. Add a Run Pipeline action to mirrored playlist cards and modals with live status polling, and make the shared pipeline lock atomic for manual and scheduled callers. |
1 month ago |
|
Broque Thomas
|
547e499121 |
Expose mirrored playlist source-ref health
Return normalized source_ref metadata from mirrored playlist APIs so the UI no longer has to infer editable refresh links from description fields. Accept Spotify embed URLs during source-ref repair and add coverage for source-ref health reporting. |
1 month ago |
|
Broque Thomas
|
73bd2db547 |
Harden playlist pipeline source refresh
Centralize mirrored playlist source reference normalization so edited links and IDs are stored consistently. Preserve URL-backed refresh refs, surface missing-source refresh failures, count background sync failures in pipeline summaries, and retry guarded automation skips after a short delay instead of losing a scheduled run. Add focused coverage for source refs, mirrored playlist source updates, refresh failures, and guarded retry behavior. |
1 month ago |
|
Broque Thomas
|
93743119d9 |
Bump version to 2.6.1
Update the SoulSync base version, Docker release workflow default, and What's New notes so the next release surfaces as 2.6.1. |
1 month ago |
|
Broque Thomas
|
0bea332aed |
Preserve album bundle track numbers
Keep album-bundle staging from replacing known per-track album numbers with the filename parser's default when staged files do not expose a real track number. Carry staging tag numbers through the cache, fall back to task metadata for private release staging, and cap hybrid album batches to one worker when Soulseek is first in the source order. |
1 month ago |
|
Broque Thomas
|
9a0e3b4011 |
Persist completed downloads in downloads view
Include a capped recent tail of database-backed download history in the unified Downloads page so completed Deezer and other streaming downloads remain visible after runtime tasks are cleaned up or the container restarts. Use persistent download history for the dashboard finished count, keep live tasks authoritative for active rows, avoid showing the local clear-completed action for persisted history rows, and cover history hydration/deduping/capping in status tests. |
1 month ago |
|
Broque Thomas
|
3ee77e3f44 |
Release 2.6.0
MINOR bump: Qobuz playlist sync is the headline feature (#677), plus the Import album search fallback-source surfacing fix (#681). * web_server.py — _SOULSYNC_BASE_VERSION → 2.6.0 * webui/static/helper.js — split the 2.5.9 'Unreleased — dev cycle' entries into a new 2.6.0 block with a real release-date marker; bumped the _getLatestWhatsNewVersion fallback default; rolled the '2.5.9 Release Stability Pass' modal section down to a generic 'Earlier in v2.5' aggregator now that 2.6.0 is the current release * .github/workflows/docker-publish.yml — bumped manual version_tag default to 2.6.0 so the next workflow_dispatch defaults right |
1 month ago |
|
Broque Thomas
|
3c0b6c6204 |
Cover missed Qobuz branches in Sync shared switches (#677)
Three follow-ups to the Qobuz playlist sync commit: * webui/static/sync-services.js openYouTubeDiscoveryModal — the syncing-phase "start polling on modal open" switch was missing the isQobuz branch (the discovery-modal-close handler hit it but this earlier hook didn't). Resuming a sync after a page refresh would have fallen through to startYouTubeSyncPolling. * webui/static/sync-services.js closeYouTubeDiscoveryModal — the per-service phase reset block had Tidal, Deezer, Spotify Public, Beatport branches but no Qobuz. After a Qobuz sync_complete or download_complete, closing the modal wouldn't reset the card phase back to 'discovered' or push the phase update to /api/qobuz/update_phase. * web_server.py _emit_discovery_progress_loop — platform_states didn't include 'qobuz', so WebSocket discovery progress broadcasts were silently skipping Qobuz playlists. HTTP-poll fallback covers it but this puts Qobuz on equal footing with the other services. |
1 month ago |
|
Broque Thomas
|
a34eae1445 |
Add Qobuz playlist sync to Sync page (#677)
Qobuz joins Tidal and Deezer as a first-class playlist sync source. New Qobuz tab on the Sync page lists user playlists + a virtual Favorite Tracks entry, and clicks route through the same discovery → sync → download pipeline the other services already use. Backend: * core/qobuz_client.py — new get_user_playlists, get_playlist, get_user_favorite_tracks, get_user_favorite_tracks_count. Returns normalized dicts (matches Deezer client shape, not Tidal's dataclasses) so the discovery worker can iterate directly without duck-typing. Virtual `qobuz-favorites` ID dispatches to favorites fetcher inside get_playlist — same trick Tidal uses with COLLECTION_PLAYLIST_ID. Both list endpoints paginate against Qobuz's 500-cap limit. * core/discovery/qobuz.py — new worker module. Mirrors core/discovery/deezer.py: pause enrichment, iterate tracks, hit discovery cache, fall back to _search_spotify_for_tidal_track, build wing-it stub on miss, sync results to mirrored playlist. * web_server.py — adds /api/qobuz/playlists, /playlist/<id>, /discovery/start/<id>, /discovery/status/<id>, /discovery/update_match, /playlists/states, /state/<id>, /reset/<id>, /delete/<id>, /update_phase/<id>, /sync/start/<id>, /sync/status/<id>, /sync/cancel/<id>. One-for-one with the Tidal + Deezer endpoint sets. Qobuz discovery executor registered for clean shutdown. Frontend: * webui/static/sync-services.js — full handler set (loadQobuzPlaylists, createQobuzCard, openQobuzDiscoveryModal, startQobuzDiscoveryPolling, startQobuzPlaylistSync, startQobuzSyncPolling, cancelQobuzSync, startQobuzDownloadMissing, rehydrateQobuzDownloadModal, etc.). Reuses the shared YouTube discovery modal via fake `qobuz_<id>` urlHash and is_qobuz_playlist flag. Shared switch statements in getModalActionButtons / generateTableRowsFromState / Wing It helpers in downloads.js gain new isQobuz branches alongside the existing per-service ones. * webui/index.html — new Qobuz tab button + content div, slotted between Deezer and Deezer Link. * webui/static/style.css — new .qobuz-icon for the tab icon. * webui/static/core.js — qobuzPlaylists / qobuzPlaylistStates / qobuzPlaylistsLoaded globals. Followed the existing per-service pattern verbatim rather than refactoring the duplicated transformers across Tidal / Deezer / Spotify-public / YouTube / Mirrored — that refactor is its own follow-up PR per the "don't break Tidal/Deezer" scope discipline. Adding the 6th copy of a proven pattern is lower risk than collapsing 5 working services behind a new abstraction. Tests: * tests/test_qobuz_playlists.py — 12 tests covering pagination, normalization, favorites virtual-ID routing, artist-name fallback chain (performer → album.artist → 'Unknown Artist'), and unauthenticated short-circuits. |
1 month ago |
|
Broque Thomas
|
de8e079a6d |
feat(media-player): playable tracks across modals + lyrics + cleanups
Three related improvements to the now-playing media player and the
"add to wishlist" / "download missing" modals.
1. Play buttons across track-list modals
Every track row in the download-missing modals (Spotify, Tidal,
YouTube, services, artist album, wishlist download-missing) and
the add-to-wishlist modal now carries a play button. Click runs
playTrackFromLibraryOrStream:
- If the track has a local file_path → playLibraryTrack
- Else POST /api/stats/resolve-track to find it in the library
by title + artist → playLibraryTrack
- Else fall back to _gsPlayTrack streaming
Backend ownership response gains track_id / title / file_path so
the wishlist modal's owned tracks can hand the right metadata
to the player without an extra round trip.
The add-to-wishlist modal previously showed the play button only
on owned tracks; now the button is unconditional so the streaming
fallback can take over for unowned ones (matches the standard
pattern from the rest of the app).
2. Clean media-player display titles
YouTube / Tidal / Qobuz / torrent / usenet plugins encode their
source-side identifier into the filename field as
<source_id>||<display> so download() can recover it later. The
media player's track-title renderer never knew about this
convention and showed strings like
"wvgFsXoGFnQ||Sometimes I Cry When I'm Alone" verbatim in the
now-playing UI. extractTrackTitle and setTrackInfo now strip the
<id>|| prefix defensively so any path into the player gets a
clean display.
Local library playback also fetches canonical metadata from
/api/stats/resolve-track when track.id is present so title /
artist / album / album art come straight from the SoulSync DB
instead of whatever the caller passed in. Falls back silently
to caller values on any error so playback never blocks on the
metadata fetch.
3. Lyrics panel + View Artist close
New collapsed lyrics panel between the playback controls and
queue panel. POST /api/lyrics/fetch (new backend endpoint)
prefers the local .lrc / .txt sidecar files SoulSync writes
during post-processing so downloaded tracks resolve lyrics with
zero network hits; falls back to LRClib exact-match (when album
+ duration are available) then to LRClib search.
Synced LRC results are parsed (handles multi-stamp lines for
repeated choruses), and the active line highlights + smooth-
scrolls into the middle of the viewport on every audio
timeupdate. Plain-text results render without highlighting.
Per-track cache prevents re-fetching when the user revisits the
same track. Lyrics fetch is fire-and-forget — failure shows
"No lyrics found" without ever blocking playback.
View Artist on the expanded player now calls
closeNowPlayingModal before navigating; the modal was previously
sitting open over the artist page, hiding it. Handler is bound
once and is a no-op when no artist_id is attached.
CSS additions are additive (new .modal-track-play-btn and
.np-lyrics-* rules); no existing styles touched. Backend endpoint
returns 200-with-success-false on any miss so callers can render
"no lyrics" without treating it as an error.
WHATS_NEW updated under 2.5.9 with two entries (lyrics + View
Artist close).
|
1 month ago |
|
Broque Thomas
|
4ebbffb898 |
Fix admin PIN after profile switches
Reset profile PIN dialog controls each time it opens so stale profile-specific event listeners cannot submit an admin PIN against a previously selected profile. Keep failed PIN attempts retryable and restrict launch-lock verification to the admin profile PIN only, so non-admin profile PINs cannot mark the admin lock as verified. |
1 month ago |
|
Broque Thomas
|
274a1ed34a |
Bump release to 2.5.9 and add changelog
Update project version and release notes for 2.5.9. Changes: update .github/workflows/docker-publish.yml default/version_tag prompt to 2.5.9, bump _SOULSYNC_BASE_VERSION in web_server.py to 2.5.9, and replace the WHATS_NEW entry in webui/static/helper.js with detailed 2.5.9 release notes and a new VERSION_MODAL_SECTIONS entry. Also update the helper.js fallback for latest whats-new version to 2.5.9. |
1 month ago |
|
Broque Thomas
|
fae13226e5 |
Check HiFi download capability via manifests
Probe public HiFi instances with the same trackManifests endpoint used by real downloads instead of the legacy /track endpoint. This prevents compatible instances from being falsely labeled search-only in Settings. Centralize HiFi instance capability checks in HiFiClient and reuse manifest URI parsing with the download path. Tests cover manifest-based capability detection, no legacy /track probe, and limited instances without a manifest URI. |
1 month ago |
|
Broque Thomas
|
6c9b43225a |
Add torrent and usenet release staging support
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file. Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely. Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels. Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests. |
1 month ago |
|
Broque Thomas
|
8b0de9eb76 |
fix(downloads): harden album bundle staging
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports. Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading. Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches. Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py |
1 month ago |